Recall-Agent

Inspiration

Most chatbots forget you the moment the tab closes. The ones that “remember” usually bolt on a second database — a vector store here, Redis there — and hope the pieces stay in sync. We wanted the opposite: one system of record that could hold a fact, a vector, and a full-text index in the same row, and decide for itself whether a new sentence is new, a revision, or a duplicate.

The working-memory loop in agents like Letta was the other spark. Open work should stay pinned (“what’s left on this ship?”). Archival facts should stay quiet until you ask. Merge and skip should be SQL, not a model guessing.

CockroachDB Cloud already spoke Postgres, vectors, and tsvector. Amazon Bedrock already had chat and Titan embeddings. The question was whether we could make that combination feel like memory, not like a demo of three APIs.

What it does

Recall Agent is a persistent-memory assistant. You start as a Guest, then claim the same user_id with a username and password or Google. Every memory query is tenant-scoped.

Chat has two memory layers:

• Open work (task_state) is pinned into the prompt every turn. • Preferences and facts are retrieved only when the model calls search_memory. Writes go through insert_memory. Finishing a job calls close_open_work and expires live task_state.

After the reply, an extractor may propose more candidates. The database, not the model, chooses ADD, UPDATE, or SKIP. UPDATE expires the old row (valid_to) and inserts a successor with a supersedes edge. /memory lets you search, inspect lineage and entities, or delete — and the next answer changes.

Retrieval is one SQL statement, not a sidecar search engine. Approximate nearest neighbors (L2), PostgreSQL full-text rank, recency, usage, and a small entity-graph hop are fused as

s = 0.55 s_(vec) + 0.25 s_(txt) + 0.10 s_(recency) + 0.10 s_(usage) + 0.12 s_(entity)

where L2 distance d becomes a similarity s_(vec) = 1/(1+d), recency is exp(−age_days/14), and usage is ln(1+hits)/ln 51.

Live demo: https://recall-agent.onrender.com/

How we built it

The app is Next.js on Render. CockroachDB Cloud is the memory layer (pg wire). Chat and embeddings are Amazon Bedrock (Claude Haiku 4.5 + Titan Embed V2, 1024-d), with an OpenAI-compatible fallback.

Schema (schema_v3.sql) puts ops, VECTOR(1024), and content_tsv on memories, with a distributed vector index prefixed by user_id and a GIN index on the tsvector. Graph edges live in memory_links. Named entities are ordinary SQL tables, hopped with a CTE — not a graph database.

A chat turn is: pin open work → at most two tool rounds → stream the reply → extract → single-transaction store. Hybrid retrieve and near-neighbor dedupe are the statements judges can EXPLAIN. Dedupe cutoffs start at d < 0.35 (SKIP) and d < 0.7 (UPDATE), and shift toward percentiles from v_l2_calibration once the extract log is large enough.

For the hackathon toolchain we used the official Cloud MCP (https://cockroachlabs.cloud/mcp) and the official CockroachDB Agent Skills repo. MCP is for operators and judges (list tables, EXPLAIN, read v_* views). The chat path never calls it.

Challenges we ran into

CockroachDB vector indexes accelerate L2 (<->), not cosine. Hybrid weights and Titan thresholds had to be rebuilt around that operator. An index hint plus JOIN q inside a larger CTE is rejected; the ANN branch has to bind $1/$2 directly and filter soft-deletes after an over-fetch.

“What’s left?” barely overlaps the stored task sentence, so archival hybrid search can miss live work. That is why task_state is pinned every turn instead of retrieved.

Older on-demand Claude 3.x IDs are often EOL on new Bedrock accounts. We standardized on inference-profile IDs (us.*) and a probe script. Free Render sleeps; the first hit after idle can take ~30 seconds.

The first live MCP EXPLAIN landed on v_memory_funnel — correctly, and with no vector index. The ANN vector search plan had to be captured on the actual hybrid statement. Keeping Cloud MCP off the request path was a product constraint, not an afterthought: that endpoint authenticates a cluster operator, not a Recall user.

Accomplishments that we're proud of

A public app where delete-in-/memory changes the next reply. One Postgres-compatible cluster doing relational ops, ANN, full-text, and a small memory graph. Versioned UPDATE instead of clobbering the old fact. SQL-decided ADD / UPDATE / SKIP, with a daily funnel view to show the write path is real. Guest → same user_id across browsers. Official Cloud MCP verified live against those views, without putting MCP on the chat path.

What we learned

Push the invariant into SQL. Models are good at proposing memories and terrible at merge policy. Index-accelerated vector search is a query-shape problem: prefix user_id, use the operator the index actually supports, bind the query vector where the optimizer can see it.

Working memory and archival memory want different access paths. Analytics views need an explicit grain — funnel is per user-day, reuse is per memory, score breakdown is per hit — or you will misread your own product.

Least privilege for judges (recall_analyst on v_* only) is easier to explain than a wide-open SQL prompt. Skills belong at development time; the agent in production should call search_memory, not a cluster-ops MCP.

What's next for Recall-Agent

Ship the three-minute public demo video. Give /memory the rest of the analytics surface judges already query over MCP (reuse buckets, score mix, entity clusters). Optionally expose Recall itself as an MCP server so other agents can use this memory store — still over pg and the user’s own user_id, never by proxying Cloud MCP on the chat path. Tune L2 cutoffs as Titan-labeled volume grows, and keep the app horizontally boring: more Next.js processes, same CockroachDB.