Unforgettable

An AI agent whose memory you can rewind, diff, and audit - versioned beliefs, AS OF SYSTEM TIME time travel, and per-reply decision audits, all as SQL rows in CockroachDB.

Comment

Inspiration

When an agent misbehaves in production, the hard question is not "what does it remember" - it is "what did it believe at the moment it acted, and where did that belief come from". A wealth-advisory bot tells a client on Monday their portfolio fits a conservative risk tolerance, then on Thursday recommends a high-volatility product. Compliance has to reconstruct the agent's belief about that client at each moment, and prompt logs only show what it said, never what it remembered.

Databases solved this for data long ago: versioning, point-in-time reads, audit trails. CockroachDB ships AS OF SYSTEM TIME - a consistent historical read served by any node. Unforgettable applies that to agent memory: every belief is a versioned SQL row, so memory can be rewound, diffed, and audited like data.

What it does

Unforgettable is a chat agent (web UI + CLI) whose entire memory lives in CockroachDB: conversation events, durable beliefs with confidence and provenance, tasks, and a per-reply recall trace.

  • Time travel: ask for the belief state at any past timestamp. Recent moments are served by AS OF SYSTEM TIME; older ones are reconstructed from the append-only version columns (valid_from / superseded_at). The API reports which mechanism answered.
  • Belief diff: beliefs are never updated in place. A repetition writes a reinforced version; a contradiction supersedes the old row and links to it through replaces_id. The diff between two timestamps lists what was learned, revised (with before/after content and confidence), and retired.
  • Decision audit: every reply records the memory rows recalled for it. Click a reply and see the beliefs behind it, and the message that taught each one.

The chaos demo makes the storage claim concrete: it starts a local 3-node cluster, holds a conversation, SIGKILLs the node the agent is using, and asserts row by row that every fact and task survived.

How I built it

Python and FastAPI over four tables: UUID keys, JSONB provenance, VECTOR(256) embeddings, append-only version columns. Retrieval is hybrid - in-database vector search plus keyword match and the conversation tail, re-ranked by similarity, recency, and keyword overlap. A consolidation job distills older episodes into beliefs. The LLM is pluggable: Claude on Amazon Bedrock (Converse API, Titan embeddings) for the hosted demo, or a deterministic zero-key client that drives the 50-test suite and the chaos demo, so everything reproduces locally in about three minutes with no accounts.

CockroachDB tools

  • Distributed vector indexing: every recall is a vector search inside the operational database. Getting that real took work - the cosine operator is never index-served on v25.2, so recall runs L2 over unit-normalized embeddings against partial prefix indexes, and a test asserts the EXPLAIN plan says "vector search" rather than full scan. Verified against the deployed CockroachDB Cloud cluster.
  • ccloud CLI: terminal-first cluster management; tools/ccloud_deploy.sh provisions a Basic cluster, SQL user, and connection URL end to end.
  • Managed MCP server: the repo ships the connect config, so the memory tables can be inspected read-only from Claude Code or Cursor.
  • Core features: AS OF SYSTEM TIME, JSONB, multi-node survivability.

AWS

Amazon Bedrock powers replies and fact extraction (Claude via the Converse API) with Titan Text Embeddings V2 for embeddings. The hosted demo runs on EC2 under an instance role - no stored keys - and reads its database credential from SSM Parameter Store. An in-app daily cap bounds LLM spend; past it, the demo falls back to the deterministic client and says so in the response.

Challenges

  • Belief semantics: "my risk tolerance is conservative" said twice should reinforce; "aggressive now" should supersede; a second stated preference must not flip the first. An explicit multi-valued subject allowlist plus tests settled it.
  • The silent full scan: an EXPLAIN plan showed the vector indexes existing but unused. The fix - L2 distance, partial prefix indexes, and a plan-asserting regression test - was the most instructive bug of the project.

What's next

Confidence decay, multi-user memory namespaces, and an MCP server of its own, so other agent frameworks can mount Unforgettable as their memory.

Built With

Share this project:

Updates