Inspiration

AI coding agents work well individually, but multiple agents on the same project often repeat work, lose context, or make conflicting decisions.

We wanted to explore a simple idea: what if agents coordinated through shared memory instead of constantly passing full conversations to each other?

That became Swarm on Memory.

What it does

Swarm on Memory is a multi-agent coding system where five agents collaborate through shared, permission-controlled memory.

  • Architect creates technical decisions.
  • Implementer sees the decisions, but not the Architect’s private reasoning.
  • Reviewer checks decisions against past context and corrections.
  • Skeptic works without shared memory to challenge assumptions independently.
  • Curator resolves conflicts and produces the final recommendation.

Agents coordinate through memory instead of directly passing responses to each other.

We also support persistent corrections, memory queries, GitHub and Notion integrations, and Memory ON/OFF comparisons for token usage, cost, and agent behaviour.

How we built it

The backend uses Python and FastAPI, with Bourdon over MCP as the shared memory and coordination layer.

Agents run concurrently using Python async workflows and have different memory access permissions.

The frontend uses React, TypeScript, and Vite, with Server-Sent Events to show agent activity, memory updates, access restrictions, and final decisions in real time.

Challenges we ran into

The biggest challenge was ensuring agents actually coordinated through memory instead of simply passing outputs to each other.

We also had to handle concurrent memory reads and writes, access permissions, multiple LLM calls, and keeping the multi-agent UI easy to understand.

Accomplishments that we're proud of

We built a system where agents genuinely coordinate through shared memory.

We are especially proud of the asymmetric memory model. The Implementer cannot access the Architect's reasoning, while the Skeptic intentionally has no shared memory at all.

We also built persistent human corrections and a Memory ON/OFF mode to compare how the swarm behaves with and without shared memory.

What we learned

We learned that shared memory can be more than storage. It can become the coordination layer between AI agents.

We also found that giving every agent all available context is not always better. Different agents can work better when they only receive the information relevant to their role.

What's next for Swarm on Memory

Next, we want to improve semantic memory retrieval, expand memory permissions across larger teams and projects, and connect the swarm more deeply with real development workflows such as issues, testing, reviews, and pull requests.

We also want to measure how much shared memory reduces repeated reasoning, conflicts, and token usage across larger sets of tasks.

Built with

Python, FastAPI, React, TypeScript, Vite, Bourdon, MCP, FastMCP, Claude, Pydantic, HTTPX, Uvicorn, asyncio, SSE, GitHub API, Notion API

Try it out

GitHub:
https://github.com/masterblaster14/swarm-on-memory

Built With

Share this project:

Updates

Submission history