total-recall-mcp

Total Recall MCP gives AI agents memory that outlives the chat. Stored in CockroachDB, found by meaning not exact words, private to you, and tough enough to survive a node dying mid-conversation.

Comment

Inspiration

AI agents forget everything the moment a session ends. For the CockroachDB × AWS "Build with Agentic Memory" hackathon, we wanted to prove that a database can be more than a cache behind an agent. It can be the actual system of record for what an agent remembers: transactional, per-user isolated, semantically searchable, and auditable enough to trust in production.

What it does

Total Recall MCP, is an MCP server that gives any MCP client (Cursor, Claude Desktop) durable memory. It exposes remember_memory, recall_memory, list_memories, forget_memory, and get_user tools. Text is embedded and stored in CockroachDB with a distributed vector index, so agents can recall memories by meaning, not exact string match. Every memory is scoped to a principal_id, so multiple users share one database with cryptographically separate memory spaces, and every single tool call, whether "success" or "failure", is written to an audit_logs table for full traceability.

How we built it

Python 3.14 with FastMCP, served over stdio for local Cursor use and Streamable HTTP for production. SQLAlchemy 2 (async) + Alembic manage the schema against a 3-node CockroachDB cluster (local via Docker Compose, or CockroachDB Cloud). A MemoryService embeds content through Amazon Bedrock's Titan Text Embeddings V2 in production, or a deterministic fake provider for zero-dependency local dev, before writing into a memories table with a VECTOR(1024) column and CREATE VECTOR INDEX ... (principal_id, kind, embedding vector_cosine_ops). A single MCPToolExecutor wraps every tool call to write STARTED --> SUCCEEDED/FAILED audit rows in their own transaction. HTTP requests are authenticated with GitHub bearer tokens, mapping each caller to github:<login>. Terraform provisions ECS Fargate, an ALB, Secrets Manager, CloudWatch, and IAM for the AWS deployment. We also installed CockroachDB's open-source Agent Skills so the agent itself has operational SQL/transaction/health expertise, alongside the Cloud Managed MCP Server and ccloud CLI for cluster inspection.

Challenges we ran into

  • A full security review turned up real architectural gaps we had to fix before this was demo-safe: the HTTP transport initially had no authentication at all, with every caller sharing one local-test-user principal — meaning tenant isolation only existed in the schema, not in practice.
  • We also found the audit trail was silently broken: FAILED rows were written before the rollback that wiped them, and successful calls never resolved past STARTED, so the "audit trail" wasn't trustworthy.
  • Separately, we were using NullPool in production, opening three fresh TLS handshakes to CockroachDB on every single tool call. And raw exception text, SQL fragments, hostnames, were leaking straight through to MCP clients. Fixing these (dedicated audit transactions, GitHub-backed auth middleware, connection pooling, sanitized error boundaries) took more time than the original feature build.

Accomplishments that we're proud of

  • A distributed vector index led by (principal_id, kind) that makes per-user isolation structural, not just application-logic, cross-tenant recall is impossible by index design, not by convention.
  • An audit trail that survives failure: every tool call leaves evidence, even when it errors.
  • Closing out a full review: 1 critical, 5 high, 9 medium, 6 low findings, before submission, rather than shipping the first-draft version.
  • Proving resilience live: killing a node in the 3-node CockroachDB cluster mid-demo, and recall still succeeds without the agent noticing.
  • One code path that runs identically against a local fake-embedding dev loop and a full AWS production stack (Bedrock, ECS Fargate, Secrets Manager, CloudWatch). No dev/prod fork.

What we learned

Auditing has to happen in its own transaction, isolated from the business transaction it's describing, otherwise a rollback erases the very evidence you need when something goes wrong. Vector index column ordering is a security decision as much as a performance one. And most of the serious findings in our security pass weren't one-line bugs. They were architecture-shaped (no auth boundary, no transaction isolation for audit writes), which is exactly why reviewing early and structurally mattered more than reviewing late and thoroughly.

What's next for total-recall-mcp-agent

Attach an ACM certificate and HTTPS listener to the ALB, Bearer tokens currently transit over plain HTTP, a known and documented demo limitation. Beyond that, memory summarization/decay policies instead of unbounded accumulation, multi-region CockroachDB to pair with a multi-region ECS deployment, and turning the existing audit_logs table into a real observability surface (invocation counts, latency, per-user activity on a dashboard) rather than a write-only trail.

Built With

Share this project:

Updates