Mnemos

final_result

Inspiration

Most "agentic memory" frameworks — Mem0, Zep, the rest — are append-only logs with a search index bolted on. They store what an agent observed and call it done. But multi-agent systems don't actually work that way in practice: a payment API says a refund went through, a support ticket says it's still pending, and now two agents believe two different things about the same order. Nobody designs for that moment. Whichever claim a retrieval call happens to surface first just quietly becomes "the truth."

That felt like the actual interesting problem. Not "can we store more context," but: when two agents disagree, what happens next? We wanted memory that treats what agents observe as claims, not facts — attributed, contestable, and actively reconciled, with the reasoning kept forever instead of thrown away the moment a decision is made.

What it does

mnemos is a memory layer for AI agents built on CockroachDB that detects contradictions and resolves them instead of quietly stacking them up.

Every write is a claim — attributed to an agent and a source with a known authority level, embedded, and checked for conflicts via vector similarity against whatever's currently canonical for that subject.
Resolution happens in stages, cheapest first: deterministic rules (source authority, recency on volatile subjects, confidence floor) run before any LLM call ever fires — an actual cost/latency guarantee, not just a README claim.
When the rules genuinely can't decide, a Bedrock arbiter reasons over both claims and returns one of four real verdict types — contradiction, refinement, temporal_shift, both_valid — not a flattened win/lose.
Ground truth outranks heuristics. For a verifiable transaction like a refund, mnemos doesn't just trust whichever source has a higher authority tier — it checks a real ledger of actual transaction state first, and that overrides the authority heuristic outright when it disagrees.
Nothing is ever deleted. A losing claim gets marked superseded, not erased — so you can search() for the one clean canonical answer, or time-travel with real AS OF SYSTEM TIME queries to see exactly what mnemos believed at any point.
The whole thing is safe under real concurrent writers, using CockroachDB's serializable transactions with explicit retry — the actual reason we built this on CockroachDB instead of anything else.

The demo scenario: a payment-agent and a support-agent disagree about a refund. mnemos resolves it with visible reasoning, a fulfillment-agent reads memory and correctly avoids a duplicate refund, and you can rewind to see the moment of resolution.

How we built it

CockroachDB Cloud for storage (VECTOR(1024) columns with a vector_cosine_ops index for the conflict-detection similarity search), Amazon Bedrock for both embeddings (Titan Text V2) and the arbiter (Nova Lite, with Claude Haiku 4.5 as a structured-output fallback), and a Python pipeline that strictly separates the two resolution stages so the "cheap rules before expensive LLM calls" claim is actually true in the code, not just asserted. AWS Lambda runs a resolution worker for the async backlog case; CDK defines all of it, including least-privilege IAM per role.

We tried to hold ourselves to "test against reality" throughout — almost nothing in this build is mocked. Concurrency correctness is proven with real threads racing the live cluster (8, then 50, then 200 concurrent writers). The AWS infrastructure was actually deployed, exercised, torn down, and redeployed from scratch, not just cdk synth-validated. The demo's reliability was measured with dozens of real Bedrock calls, not assumed from a couple of manual runs.

Challenges we ran into

The flagship demo's own conflict detection was flaky — real embedding similarity for "refund pending" vs "refund processed" sometimes landed on the wrong side of the classification boundary, and a first attempt at fixing it (lowering the threshold) closed the gap but didn't close it completely. The actual root cause turned out to be one ambiguous line in a prompt — "lightly clean up vs. rewrite" — that let the model inconsistently pass source jargon through unnormalized, occasionally producing an outlier embedding. Fixing the prompt, not the threshold, is what actually closed it (verified 20/20 afterward, with a real margin instead of a coin flip).

Deploying to AWS for real surfaced a bug that looked exactly like a network problem — every Lambda invocation timed out with zero log output, which looked suspiciously like a CockroachDB Cloud IP allowlist blocking Lambda's egress. It wasn't. It was pgvector handing back a non-iterable Vector object on a code path that, unlike everywhere else in the app, re-read an embedding out of the database instead of generating it fresh. Would have been very easy to misdiagnose and chase the wrong fix for hours.

And a genuinely interesting one, pointed out directly during review: a strict authority-tier cascade can't tell the difference between "authority proxy" and "actual truth" — a stale claim from a trusted source still beats a fresh, correct one from a less trusted source, every time, no matter how confident the correct claim is. That one didn't have a config fix. It needed a new resolution stage.

Accomplishments that we're proud of

Zero lost updates under 200 real concurrent writers against CockroachDB's default serializable isolation — while the identical naive transaction logic against a real local Postgres instance silently lost 163 of 200 updates with no errors at all. That comparison is the entire pitch for using CockroachDB here, measured, not marketed.

Adding ground-truth verification after the fact, without breaking anything: we proved it's a genuine override — not a coincidence — with a test where the lower-authority source's claim wins specifically because a real ledger confirms it and the higher-authority claim's version doesn't hold up. That's the moment the system stopped being "weigh two opinions" and started being "check what's actually true when you can."

Keeping the documentation honestly self-critical the whole way through — every real bug, every partial fix, every "this doesn't fully generalize yet" is written down, not smoothed over for the pitch.

What we learned

A deterministic rule cascade is easier to explain and audit than a weighted score, and that's a real, deliberate tradeoff worth defending — but it means "authority" is only ever a proxy for trust, and a system that can't occasionally check real ground truth will confidently be wrong sometimes. Hard similarity thresholds are inherently fragile near their boundaries no matter where you set them; the fix that actually holds up is reducing the variance feeding into the comparison, not endlessly retuning the cutoff. And "test against reality" is expensive in wall-clock time — real Bedrock calls and real cross-region DB round trips make every verification loop slow — but it's the only way we caught bugs that would have looked completely fine in a mocked test suite.

What's next for Mnemos

Generalizing ground-truth verification beyond refunds — giving the arbiter real, scoped tool-access to check other verifiable domains, instead of one hardcoded ledger table for one subject type. Giving both_valid a real place in the schema instead of being the one verdict type that doesn't cleanly fit today's resolutions table. Replacing the EventBridge-polled Lambda with a real CockroachDB changefeed for near-real-time resolution instead of a fixed-interval sweep. And running everything under a properly scoped IAM identity instead of the account root user we used to get the deploy cycle verified under deadline pressure.