Hivemind

Inspiration

We started with a simple question:

Why are people expected to adapt to technology instead of the other way around?

The web has made everyday tasks more accessible in theory, but using it still requires people to understand interfaces, menus, buttons, and website-specific workflows. We were interested in whether AI agents could remove some of that complexity by letting people describe what they want rather than how to navigate a website.

That led us to a second problem: AI agents themselves are constantly relearning how to use the web.

If one agent spends time figuring out how to navigate a website, another agent encountering the same task may have to repeat the entire reasoning process. Humans don't work this way. When we learn how to do something repeatedly, we remember it, turn it into a shortcut, and teach other people.

We wanted to give AI agents that same kind of shared procedural memory.

What We Built

We built Hivemind, a shared, self-compiling memory system for browser-automation agents.

An agent can receive a task in plain English, such as:

"Log in and add the Sauce Labs Bolt T-Shirt to the cart."

The agent uses an LLM to navigate a real browser and complete the task. While it works, Hivemind records every action the agent actually performs — including clicks, navigation, and typed values.

When the same task succeeds twice with an identical action sequence, Hivemind automatically compiles those interactions into a deterministic script.

From then on, instead of asking an LLM to reason through the task again, the system can replay the learned script directly. This means future executions require zero LLM calls, can complete in under two seconds, and are verified against the actual outcome of the page.

Most importantly, the learned skills are shared across a team. One person's agent can discover a workflow, and another person's agent can immediately reuse it.

In other words:

One agent pays the cost of learning. Everyone gets the shortcut.

How We Built It

Our system is built around three main components:

An LLM-powered browser agent that understands natural-language tasks and interacts with real websites.
A self-compiling skill library that records successful action sequences and turns repeated workflows into deterministic scripts.
A shared hub and dashboard that lets multiple users and agents access the same library, activity feed, groups, and learned skills.

We used TypeScript, Node.js, Express, Playwright, Steel.dev, and the Anthropic SDK, with support for local models through LM Studio.

The core execution loop is:

Explore → Record → Verify → Compile → Share → Replay

If a learned script stops working because a website changes, Hivemind can fall back to LLM-based exploration rather than blindly repeating a broken workflow. A new successful workflow can then replace or improve the old one.

What We Learned

One of our biggest takeaways was that browser automation is much harder than simply telling an LLM to click buttons.

We initially expected browser traces to give us everything we needed to build reusable scripts. However, Steel's session traces redact typed values and can contain inconsistent or unexpected browser events. This meant we couldn't reliably reconstruct the exact sequence of actions from the trace after the fact.

Instead, we made the agent itself the source of truth: every action is recorded at the moment the agent performs it. This gave us an exact representation of what actually happened.

We also learned that successful browser automation cannot be determined solely by whether an operation throws an error. A form submission can fail silently and simply display a validation message. Hivemind therefore verifies the actual final page state, including the expected URL, before considering a replay successful.

Another challenge came from browser interaction itself. Coordinate-based clicking through Playwright was unreliable over Steel's CDP connection, sometimes reporting success even when the click landed incorrectly. We solved this by using native DOM element clicks rather than relying on coordinate calculations.

Challenges

The biggest challenge was making the system trustworthy enough to turn an exploratory AI interaction into a deterministic automation script.

We had to solve several problems:

Recording actions reliably rather than trying to reconstruct them from imperfect traces.
Determining when a workflow is safe to compile, requiring repeated successful executions with an identical action sequence.
Making replay reliable across real browser navigation and JavaScript execution contexts.
Verifying actual outcomes instead of assuming that no exception means success.
Sharing learned skills across multiple users and agents while keeping groups and their data isolated.

These challenges taught us that the interesting part of browser agents isn't just making an LLM capable of using a website. It's building the infrastructure that allows an agent's experience to become reliable, reusable knowledge.

What's Next

Hivemind is a first step toward a web where agents don't have to rediscover how to use the internet every time they encounter it.

Today, a successful interaction can become a reusable script. In the future, we want this shared memory to become much more expressive: agents could learn from failures, combine skills to solve more complex tasks, and continuously adapt as websites change.

Our broader vision is simple: