-
-
Meet Kiwi: support that lives on your screen.
-
Start where onboarding hurts.
-
A slow start has a real price.
-
New hires don't fail at the job. They get lost in the tools.
-
Every generation of support made you describe the screen.
-
Without Kiwi: tickets and waiting. With Kiwi: you're taken there.
-
Kiwi follows the question across apps.
-
One sentence in. One answer out.
-
Fast, and it follows your rules.
-
Onboarding is just the start.
A new hire needs about 65 days to reach full speed, and works at only about 25% for the first 30. For people learning complex applications, poor onboarding also doubles the odds that they start looking for another job. Most of that time is not spent learning the job itself. It is spent learning where things live in the tools: which list holds the orders, which menu finishes a setup, which rule blocks a quote. Help usually means a ticket, a Slack thread, or waiting on the one person who knows the tool. Every generation of support, from phone calls to chatbots to AI calls, still made the user describe their screen. We wanted support that already sees the screen and already knows the company.
What it does
Kiwi is a desktop guide for enterprise applications and internal tools. You hold a key and say what you need, and Kiwi looks at your screen, understands the request, and either points you to the right control or does the clicking with you. What makes it different from a generic computer-use agent is that it decides and acts using rich, enterprise-specific context. It knows who you are, what your role is, who your manager is, and which team rules apply. So it can explain why something is blocked, not just where the button is.
Kiwi can also record skills. A skill is a sequence of actions, a workflow, that an experienced employee performs once while Kiwi watches. Kiwi saves it, and anyone on the team can later ask Kiwi to run or walk them through that workflow. Experienced people record team-specific know-how once, and juniors can reuse it, learn from it, and ramp up much faster.
How we built it
- LangChain deepagents at the core. Kiwi's brain is a LangChain deepagent, built on LangGraph. It gives us planning, tool use and state management out of the box, so we could spend our time on enterprise context instead of agent plumbing. The agent plans a task as a sequence of steps and uses tools to look up who you are, your company's rules, and the pages your company wrote, before it decides where to point or what to click.
- Planner and executor loop. We built a producer-consumer pattern on top of deepagents. A planner agent writes plans to a buffer, speculating up to three steps ahead, and an executor agent works through the front of the queue in parallel. If the current step does not match what is actually on screen, the executor flushes the buffer and the planner recomputes the plan. This repeats until the goal is reached.
- Tools for the agents. We expose tool calls that let the deepagent work directly with the knowledge base. It can follow relationships, look up enterprise context such as team rules and documents, and ground each step in what the company actually does.
- Smart routing. Simple commands go to a single Gemini API call. Complex or multi-step commands go to the deepagent. Requests with question-like semantics, such as "show" or "what", are routed to the lighter path, which keeps simple answers fast.
- Knowledge graph and retrieval. We use Neo4j to build a knowledge graph of enterprise context: roles, employees, reporting lines, company documents, FAQs, and other unstructured data. We store document embeddings in Pinecone for retrieval-augmented generation, so Kiwi can pull the pages a company actually wrote and answer with better context.
- Desktop app. The app is built with Electron. We use Electron's
ipcMainfor the computer-use layer, screen capture, and screen recording, and AssemblyAI to turn voice into commands.
How a company adopts Kiwi
Kiwi runs on a company's own context, and most of that context is unstructured: PDFs, emails, spreadsheets and wiki pages. Our plan is to send forward-deployed engineers (FDEs) into each customer. An FDE takes that unstructured data and turns it into the company's knowledge graph and retrieval index. Each customer gets their own graph, and Kiwi then answers and acts from it. That makes the deployment model repeatable: new customer, new graph, same agent.
Challenges we ran into
- Agent orchestration across multi-page workflows. Real tasks span several pages, and a page change in the middle of a plan can interrupt it. Keeping the planner and the executor in sync, and recovering when the screen did not match the plan, took the most iteration.
- Context management and knowledge base traversal. Putting everything into the prompt is slow and noisy, and calling tools for everything is slow too. We had to decide what to load up front, what to fetch on demand, and how to walk the graph to the right facts without wasting time.
Accomplishments that we're proud of
- Knowledge base traversal and content prefetching. Kiwi finds the context it needs quickly and loads likely context ahead of time, so common questions do not wait on a lookup.
- Agent orchestration on LangChain deepagents. A planner and executor loop on top of deepagents that recovers when the screen does not match the plan, combined with a knowledge graph the agent can traverse as a tool.
- Reusable skills. Recorded workflows that anyone on the team can replay or ask Kiwi to walk them through.
What we learned
- Computer use: how to connect a model to a real screen, read it, and act on it safely, including when to point and when to click.
- LangChain deepagents: how to structure planning, tools and state so an agent can recover from surprises, and how to extend the framework with our own planner and executor loop.
- RAG: how much the quality of retrieval and context decides the quality of every answer, more than the choice of model.
What's next for Kiwi
- Denser knowledge bases. Deeper knowledge bases can grow into dense ontology graphs that turn unstructured enterprise data into structured knowledge, with only minimal tweaking from engineers. That structured context is what powers Kiwi.
- A repeatable FDE playbook. Package how our forward-deployed engineers turn a customer's unstructured data into a knowledge graph, so each new deployment takes less time.
- Wider reach. With that foundation, we can streamline onboarding and custom support for most enterprise applications, then extend to sales ramp, education, and any tool with a screen.
Built With
- ai-harness
- deepagent
- electron
- geminiapi
- knowledge-graph
- langchain
- neo4j
- pinecone
- pipelines
- rag
- typescript
