Clara

Elevator pitch

Clara is the agent that lives on your machine and gives every other AI agent a body.

Inspiration

There's a swarm of AI agents out there now, each one good at one narrow thing. But none of them actually live anywhere. We wanted an agent that sits on your own computer, has context and control over your machine, and can pull in any of those external agents and put them to work for you.

What it does

Clara gives you full access to your laptop through iMessage (via Poke). You text it like you'd text a friend: order groceries on Amazon, book a table on Yelp, whatever.

Behind the scenes, a classifier figures out what you actually want, builds a plan, and hands it off to a specialist agent in the swarm. For Amazon orders, Clara builds the cart and shows it to you in chat before doing anything. You approve or reject, then it executes. For restaurants, you get a list of options right in iMessage, pick one, and Clara finishes the reservation on your desktop.

Under the hood, this is a real swarm of agents running on Fetch.ai's Agentverse, not one bot pretending to be many.

How we built it

Three pieces:

An Express + MCP middleware pipeline that talks to Poke
A Fetch.ai uAgents orchestrator that handles intent classification and routing
An Electron app on the desktop that handles automation and cowork

Intent classification runs through ASI:One's LLM. There's a dashboard with access to the Agentverse marketplace, so you can connect any agent you want into the swarm, plus a dummy fallback agent so the demo never dies if ASI is down. The orchestrator routes to downstream specialist agents over the Agent Chat Protocol. Redis backs state and trust scoring, with a disk fallback, so the system always has context on what's happening on your desktop. MCP runs over StreamableHTTP so Poke can talk to it directly.

Challenges we ran into

Bridging a text thread with a desktop. iMessage is built for sending words back and forth, not for telling a laptop what to do. Getting Poke to talk to the Electron app cleanly, so a casual text actually triggers something real on your machine, took more wrangling than it looked like it should.

Giving the AI enough context about your desktop without giving it everything. The agent needs to know what's going on on your machine to act on your behalf, but dumping your whole desktop at it is messy and a little unnerving. We had to figure out what context actually mattered and feed it just that, without losing the thread of what you're doing.

Keeping multiple agents talking without crossing wires. Once you've got a swarm instead of one bot, you have to make sure a reply from the Amazon agent doesn't get matched to a restaurant request that came in five seconds later. Sounds simple until two requests are in flight at once.

Making sure a demo never just dies. Any one piece, ASI, Redis, the executor, could go down mid-demo. We built the whole thing to fail soft instead of fail loud, so even if something upstream broke, Clara still looked like it worked.

Accomplishments that we're proud of

Plug in any agent from the Agentverse marketplace, not just the ones we built. Clara isn't locked to a fixed set of skills. Open the dashboard, connect an agent from the marketplace, and it's part of your swarm. The system was built to grow past Amazon and Yelp from day one.

Hands free control of your desktop with voice. You can talk to your machine like you'd talk to a person and have it actually do something, no keyboard, no clicking through menus. That's the part that made people in the room go quiet for a second.

A purchase never happens without you seeing it first. Even with agents acting on your behalf, nothing gets bought or booked blind. Clara shows you the cart, you say yes or no, and only then does it execute. We wanted autonomy without giving up control.

What we learned

On working with Fetch.ai/uAgents and ASI:One: The agent protocol doesn't hand you anything like a request ID to match a reply against. We assumed agent-to-agent communication would feel like calling an API and getting a response back. It's closer to mailing a letter and hoping it comes back in the order you sent it. ASI:One's classification itself worked well out of the box. The surprise was how much hand-built plumbing it took to make agent replies land back in the right place.

On giving an AI control over your desktop: Most software adds a button. This adds judgment. The hard part wasn't teaching Clara to act on your machine, it was deciding what she should never be allowed to do without asking first. Control turned out to be a design problem, not an engineering one.

On the word "agentic" itself: Building an actual swarm of agents made us a lot more skeptical of how that word gets used in pitch decks. The real work is unglamorous: matching replies to requests, building fallbacks for when a service goes down, deciding who owns state when three agents are talking at once. None of that is exciting to demo, but it's most of what makes the thing actually work.

What's next for Clara

More agents, more use cases. Amazon and Yelp were the proof of concept. The marketplace model means the next agent someone plugs in could be anything, travel booking, calendar management, your own weird side project. The sky's the limit once the swarm is the product instead of the two skills we shipped.

Open sourcing the connector. We want anyone to be able to build their own specialized agent and wire it straight into Clara, instead of waiting on us to add it. That's the actual unlock here. Clara stops being our project and becomes whatever the community plugs into it.

Beyond iMessage, and real payments. Texting was the fastest way to prove the idea worked. Long term, more channels and an actual payment flow, instead of a cart you just approve and someone else checks out, turns this from a demo into something you'd trust with real money.