Memory Universe

Inspiration

Everyone on my team works with an AI agent now, and none of those agents can see into anyone else's. So we still write the doc, and then we sit in the meeting to explain the doc. Shared memory should fix that — one place the whole team's context lives — but almost nobody turns it on, and I don't think the reason is technical.

Every agent-memory product I looked at has the same shape: one store, everything in it, and a WHERE clause deciding what you're allowed to read. Your private notes physically sit on the same server as your team's, in the same index, and the thing protecting them is a filter. One filter bug, one over-broad recall, one leaked token, and "private" was never private. So people keep their memory siloed per person, and the team never gets the compounding value of shared context.

I wanted to find out what changes if you make the privacy boundary a physical thing instead of a filter. Not a stronger WHERE. A different machine.

## What it does

Your private memory lives in your own local Redis, Qdrant, and FalkorDB, on your machine, and is never written to the shared server at all. Shared and team memory live on a hosted server. "Private" isn't a permission you're trusted to respect — it's a different place. You can forge an auth header against the shared server all you want; the private memory isn't there to find.

On top of that boundary it's a real memory system: three tiers that promote and demote, bi-temporal facts where a correction supersedes the old one instead of piling up next to it, deterministic policy authorization that a model can propose against but never override, and durable local-to-shared sync through a Temporal worker. Any agent connects over MCP — I have Codex write a memory and Claude Code read the exact same record back from the same session.

## How I built it

Python, FastAPI, FastMCP, Redis, Qdrant, FalkorDB, Temporal, Azure OpenAI, a local MiniLM embedder, and a React + Vite client. Two physically separate Compose store sets, local and shared, so the boundary is real in development and not just in a
diagram.

I built it from an empty repository during Build Week, with Codex on GPT-5.6 driving the whole thing. I decomposed the work into sub-agents, each handed a brief naming the exact files it owned and the files it must not touch, isolated in its own git worktree, test-first, with ruff and mypy --strict gating every merge. The shipped system routes GPT models by role — gpt-4.1-mini for light classification, gpt-4o for routine extraction, gpt-5-chat for answering and the hard calls — through one provider-neutral router. GPT-5.6 is the build-time tool here, not a runtime dependency, and I kept those two strictly separate in the docs.

The best thing Codex did wasn't writing code. I pointed it at my own proof harness and asked it to audit it — and it found nine real defects in the thing I trusted most.

## Challenges I ran into

Proving an absence is much harder than proving a presence. My privacy claim rests on a six-proof gate that reads Redis, Qdrant, and FalkorDB directly instead of trusting the app's success flags — and Codex's audit found that four of those proofs could never have failed. Absence checks with no positive control couldn't tell "correctly isolated" from "the write silently no-op'd." Every leak check was keyed on a memory ID, so a private note copied verbatim into a shared record under a fresh ID would have sailed through. I fixed each one and then broke every proof on purpose to watch it go red.

Real infrastructure surfaces bugs that fakes can't. The first live run died on a Redis parser that never consumed a trailing CRLF — invisible until you read a real multi-item reply.

## What I learned

A green check is worth exactly as much as its ability to go red. And a claim nobody measured is a claim nobody should have written — "GraphRAG" sat in my architecture doc describing a function that split a string on whitespace, and it survived only because every test asked whether data landed, never whether a question got it back. The moment I wrote the experiment that asks, it took four minutes to disprove. I deleted the claim.

## What's next

It's an MVP and I'm blunt about the gaps: the retrieval numbers come from a small hand-labelled probe, not a public benchmark, and auth is a demo stub. Next is real auth terminated at a proxy, local extraction so private content never reaches a retrieval the local path already uses.

Built With

• azure-openai
• codex
• docker
• docker-compose
• falkordb
• fastapi
• fastmcp
• gpt-4.1-mini
• gpt-4o
• gpt-5-chat
• gpt-5.6
• mcp
• minilm
• pydantic
• pytest
• python
• qdrant
• react
• redis
• sentence-transformers
• temporal
• typescript
• uvicorn
• vite