Walkable

Inspiration

Learning complex desktop apps feels broken: tutorials are generic, docs are scattered, and users get stuck at the exact moment they need context. Walkable was inspired by the idea of an AI teacher that can see the app, understand the user’s goal, and guide them step by step inside the real workflow.

## What it does

Walkable is a macOS learning agent for desktop software. A user says what they want to learn, like making a beat in Ableton, and Walkable creates an interactive course, inspects the live app UI, highlights where to click, answers questions, researches docs/web sources, and adapts the lesson as the user progresses.

## How we built it

I built a Swift macOS overlay app with Conduit for agent/tool calling, a OpenAI comptaible model for model access, accessibility APIs for app inspection, TinyFish for web research, Senso for ingested app documentation, and Ghost Postgres for saving reusable research runs.

Challenges we ran into

The hardest parts were making the agent useful inside a live desktop app, keeping tool calls visible and debuggable, grounding guidance in real app state, and wiring multiple sponsor APIs into one coherent research flow without turning the product into a pile of demos.

Accomplishments that we're proud of

We’re proud that Walkable feels like an actual product: it can launch apps, inspect UI, guide users through real workflows, research official docs, show its tool calls, and persist research memory. It is not just a chatbot next to an app; it is a contextual teacher inside the workflow.

What we learned

We learned that desktop learning agents need three kinds of context: live UI state, trusted documentation, and web research. We also learned that visible tool calls make the experience much easier to trust, especially when the agent is taking actions or doing deep research.

What's next for Walkable

Next we want to make Walkable work across more apps, improve course memory, add richer voice interaction, deepen MCP/tool integrations, and let each app-specific research agent manage its own documentation, data shape, and saved learning paths.

Built With

• openai
• senso
• swift
• tinyfish