colony8

Transactional shared memory for multi-agent fleets: every write resolves ADD/UPDATE/NOOP/SUPERSEDE inside a serializable CockroachDB transaction, so no agent ever acts on contradicted state.

Comment

colony8 — transactional shared memory for multi-agent fleets

One sentence: colony8 gives a fleet of stateless AI agents a single CockroachDB memory where every write is resolved inside a serializable transaction — so no agent can ever act on a stale or contradictory copy of shared state.

Inspiration

37% of multi-agent system failures trace to inconsistent shared state (Cemri et al., summarized by O'Reilly, 2025). One agent writes, another reads a snapshot that is already wrong, and both keep going. Databases solved this class of bug decades ago; agent frameworks mostly re-invented in-memory caches instead. We wanted to see what happens when agent memory is treated as what it actually is: a concurrency problem.

What it does

Stateless agents on Amazon Bedrock — a planner and N researchers — share exactly one memory. Every candidate finding walks the same pipeline before it can land:

candidate ──► 1 SNAPSHOT ────────► 2 CLASSIFY ─────────► 3 APPLY
              vector recall,        LLM verdict:           one SERIALIZABLE txn
              no lock held          ADD · UPDATE ·         SELECT ... FOR UPDATE
                  ▲                 NOOP · SUPERSEDE       + version fence
                  │                                            │
                  └── re-snapshot, re-classify ◄── version drift / SQLSTATE 40001
                                                               │
                              retries exhausted ──► DEFERRED (parked, never dropped)

SUPERSEDE never deletes: the losing claim is retired with invalidated_at and superseded_by set, so the full contradiction chain behind every live fact stays queryable forever.

Memory also outlives the process. When a new session asks a related question, the colony's live claims are injected into its researchers' context before they touch a single source — recorded on the ledger as an auditable INJECT event:

 session 1 · "thermal properties?"          session 2 · "cooking at altitude?"
 planner + 3 researchers                    new process, new fleet
       │ writes · 90C SUPERSEDED by NIST          ▲
       ▼                                          │ INJECT · prior claims enter
 ┌────────────────── one memory ─────────────┐    │ researcher context; nothing
 │  CockroachDB · SERIALIZABLE · chains kept │────┘ is re-learned
 └───────────────────────────────────────────┘

The 2:39 video shows all of it live: the 90C claim struck through on screen as NIST lands, the backend killed and restarted with nothing lost, and a brand-new session inheriting the colony's knowledge at the exact moment it becomes relevant.

How we built it

CockroachDB carries the whole memory: two C-SPANN vector indexes (one run-scoped for the write path, one colony-wide for cross-session recall), serializable transactions for the resolver, and the Cloud Managed MCP Server as the only access path for the memory-audit agent. Bedrock runs the planner, researchers, the op classifier and audit narration through the Converse API — Nova Pro on the deployed demo; any Claude id drops in via BEDROCK_MODEL_ID — with Titan v2 embeddings. The API ships as a Lambda container image behind API Gateway; the ledger UI is React on S3 + CloudFront; scripts/provision_cloud.sh scripts the provisioning via ccloud.

The numbers

Substrate (laptop, single-node CockroachDB; LLM calls excluded):

operation throughput p50 p95
write, fast-path ADD ~135 op/s 7.6 ms 8.3 ms
semantic recall, k=5 ~395 op/s 2.3 ms 3.7 ms
200 conflicting writes, 8 threads converge to 1 live fact · 0 lost writes

Then we benchmarked the design decision — 246 contradicting writes over 100 facts in nine contradiction shapes, against reimplementations of the resolution algorithms Zep/Graphiti, Mem0 and Letta document publicly (same store, embedder, classifier; no vendor code run — these are not their products' scores):

resolution algorithm final answer stale rows in context values lost
Zep/Graphiti-style · bi-temporal 94% 0.1 0
colony8 · write-time, fenced 88% 0.1 0
Mem0-style · in-place ops 88% 0.1 222
vector store · append-only 88% 1.1 0
Letta/MemGPT-style · block rewrite 79% 0.2 198

colony8 does not win this table, and we shipped it anyway. Zep's bi-temporal model records when a fact was true, not just when it was written, and that single capability is the entire 6-point gap. A memory layer that hides its failure modes is the thing this project argues against.

Challenges we ran into

  • CockroachDB's vector index can be silently decorative. The wrong opclass, or any WHERE predicate on the ordering query, drops you to a full scan with no warning — EXPLAIN is the only witness. We rebuilt recall as index-first with an exact filtered fallback so correctness never depends on the index.
  • Our own benchmark lied to us three times — a seeding race, a nearest-neighbour classifier artifact, accidentally monotonic confidences — before a control group of never-contradicted facts (where every algorithm must score identically) pinned it.
  • Shared memory makes test hygiene a correctness concern: once recall spans every run, leftover benchmark facts get injected into real sessions. The harness had to become self-cleaning.

Accomplishments we're proud of

The race is not hand-waved: test_concurrent_contradiction and the bench run live in the video — 200 conflicting writes converge to one live fact with zero lost writes and every loser auditable. And the comparison table above includes the row where a competing design beats us, with the reason and the fix on the roadmap.

What we learned

Write-time resolution does not buy accuracy — recency ranking matches our 88%. What it buys is everything around the answer: 0.1 stale rows per retrieved context instead of 1.1, one live row per fact instead of four, zero lost values instead of ~200, and an audit chain. The most useful benchmark was the one we lost.

What's next

  1. Bi-temporal facts — valid_at/invalid_at on findings, event time deciding what is current. Closes the measured 6-point gap; the supersede chain already has the right shape.
  2. Source authority for ties — two disagreeing sources with no ordering defeat every algorithm we tested (0% across the board).
  3. Fence concurrent creation of brand-new near-identical claims.
  4. A credentialed head-to-head — run Mem0 and Zep as themselves with a pinned embedder, reporting Recall@10 alongside the resolution metrics.

Tool identification

  • CockroachDB Distributed Vector Indexing — two C-SPANN indexes (vector_cosine_ops): a run-prefixed findings_embedding_idx serves the semantic recall that runs before every write and feeds candidate matches to the resolver, and an unprefixed findings_embedding_colony_idx serves the colony-wide recall that injects prior sessions' claims into new researchers. EXPLAIN on the shipped query shows • vector search against the index.
  • CockroachDB Cloud Managed MCP Server — the memory-audit agent's only access path. It introspects the fleet's own memory over read-only SQL via MCP and produces the narrated health report (contradiction rate, supersede counts) attached to a run when CRDB_MCP_TOKEN is set; the hosted replay runs were populated without it, so their ledger stats come from the ledger API directly.
  • ccloud CLI — scripts/provision_cloud.sh creates the free-tier cluster and SQL user and prints the connection string, so provisioning is scripted and repeatable.
  • Amazon Bedrock — the Converse API for the planner, the researchers, the ADD/UPDATE/NOOP/SUPERSEDE classifier and the audit narration (Nova Pro on the deployed demo; Claude ids drop in via BEDROCK_MODEL_ID once an account's Anthropic use-case form is approved); Titan Text Embeddings v2 for the vectors the index stores.
  • AWS Lambda + API Gateway — container-image runtime for the FastAPI surface and ledger replay, fronted by an HTTP API.
  • Amazon S3 + CloudFront — static hosting and CDN for the React ledger UI.

Feedback on tools

Vector indexing was the sharpest edge. An index created with the default opclass is silently ignored by a query ordering on <=> — no error, no warning, just a full scan — and adding an ordinary WHERE predicate to the ordering query defeats it the same silent way. EXPLAIN is the only way to find out, so it is worth checking for • vector search in the plan rather than assuming the index is doing work. The Managed MCP Server was the opposite experience: pointing an agent at it and getting read-only SQL introspection took minutes, and constraining the agent to that one path made "the fleet audits its own memory" a genuinely small amount of code.

Built With

Share this project:

Updates

Submission history