Recall — User-Controlled AI Memory

Instead of giving an agent access to your memory, Recall gives your memory a controlled interface the agent must use.

Comment

Inspiration

I was sitting in a coffee shop in Lagos, half-watching a friend talk to ChatGPT on his phone. He asked it, "What do you remember about me?" ChatGPT rattled off his job, his preferences, and a trip he'd mentioned weeks earlier. He laughed, then paused: "But I never told it to remember any of that."

That moment exposed a problem that only gets more important as AI agents become more capable: your AI can remember things about you, but you have almost no control over the memory itself. You can't inspect individual facts, edit them precisely, audit when a memory was used, selectively revoke access, or independently verify what actually happened.

When I read the WebMCP specification, I saw a fundamentally different way to approach this. Most agent architectures put the agent at the center — the agent calls out to servers, and the server owns the capability surface. WebMCP inverts that relationship: it lets a website publish its own state as a tool surface that an external agent can access through the browser. So I inverted the architecture around that idea. Instead of giving an agent access to your memory, give your memory a controlled interface the agent must use.

That became Recall: the website is the memory, the agent is the consumer, the browser is the trust boundary, and the user controls the capabilities.

What It Does

WebMCP changes the boundary. Traditional agent memory is an API the agent consumes. Recall makes the website itself the memory boundary and publishes only the capabilities the user has enabled.

Recall is a user-controlled memory layer for AI agents, implemented as a WebMCP website. Your AI's memory of you lives in a website you can inspect, edit, control, and audit, instead of disappearing into an opaque agent platform.

The clearest demonstration of that is this sequence:

ChatGPT
   ↓
discovers Recall's WebMCP tools
   ↓
addFact("I prefer TypeScript")
   ↓
Recall updates the memory
   ↓
signed audit event appears
   ↓
user disables addFact in Settings
   ↓
agent attempts a write
   ↓
❌ blocked by Recall
   ↓
user re-enables addFact
   ↓
✅ access restored

Disable the addFact tool in Settings, and the capability token's scope narrows immediately. Ask the agent to remember that you prefer Rust, and it can no longer perform the write — the call simply fails. Re-enable addFact, and access returns just as quickly. In one sequence, this demonstrates WebMCP capability exposure, user governance, authorization enforcement, and agent behavior in a directly observable way. The website isn't only exposing capabilities to an agent; it is actively enforcing the user's decision at the capability boundary in real time.

Under the hood, when a signed-in user opens Recall in a WebMCP-capable browser, it registers six tools through document.modelContext.registerTool():

Tool Purpose
query Retrieve matching memories using content and tag search
addFact Add a memory
updateFact Update a memory
forgetFact Soft-delete a memory
summarize Produce a deterministic summary
timeline Retrieve recent agent activity

These tools carry WebMCP annotations such as readOnlyHint and untrustedContentHint, and Recall also demonstrates origin-scoped access, declarative tool exposure, browser-mediated authentication, and capability-token authorization.

Every action in the sequence above also leaves a receipt. Each tool call is signed with ECDSA P-256 and appended to a tamper-evident audit record, then made available as an exportable JWS bundle. Anyone can independently verify, using the included public key, that a given audit event was signed by Recall and has not been altered since.

Recall is designed for people who increasingly rely on AI across multiple agents and need their memory to remain under their control rather than becoming locked inside a single agent provider. As context, preferences, and history become distributed across ChatGPT, Claude, and future agents, the question becomes not only how AI personalizes itself, but who owns and governs the persistent context behind that personalization.

Recall's answer is portable, governed, auditable personal memory for the agentic web: memory that can outlive any one agent relationship because the user, not the platform, holds the capability boundary. The distinctive part of Recall is not simply "user-controlled AI memory" as a category. It is that the memory itself is exposed as an agent-facing WebMCP capability boundary, with real-time capability revocation and independently verifiable action receipts.

Beyond the core protocol, Recall is implemented as a complete product: a memory canvas with full CRUD operations, tag-aware search with a graceful fallback when nothing matches, a Memory Insights dashboard, a real-time activity feed, per-tool permissions, short-lived capability tokens, a signed audit export, an interactive WebMCP playground, a diagnostics page, and an in-app tool-call simulator. The goal was never to maximize the number of AI components; it was to build the smallest system that makes user-controlled agent memory real and demonstrable.

How We Built It

The architecture follows the inversion deliberately:

Traditional Agent Architecture Recall
Subject Agent Website
Client Server ChatGPT
Tool surface Server TLS origin
Trust boundary Network Browser
Memory owner Agent platform User

Recall is built with Next.js 16 and the App Router, TypeScript, WebMCP's document.modelContext, Turso/libSQL for edge-replicated SQLite, WebCrypto for signing, socket.io for real-time updates, TanStack Query on the client, and Vercel for deployment.

We use GitHub OAuth for browser-mediated authentication. The /playground page offers a no-sign-in interactive tour of the tool surface (using a simulated vault), while /app provides the full WebMCP experience with real tool registration, the signed audit log, and per-tool permission controls. The stack is intentionally restrained: there is no blockchain, multi-agent orchestration, or RAG pipeline in the MVP because none of these would strengthen the core product. We deliberately optimized the system for verifiability and user control rather than making its architecture dependent on opaque retrieval behavior.

The security model treats WebMCP's browser topology as the foundation rather than a layer bolted on afterward. TLS-origin tool publication, browser-mediated execution, fromOrigins restrictions, per-tool permissions, short-lived capability tokens, audience and scope restrictions, ECDSA signatures, and tamper-evident audit records all work together. The user can disable any of the six capabilities; the WebMCP bridge only exposes the enabled set, and each capability token is scoped to the current permission state. Revocation is therefore part of the actual execution path, not simply a UI preference.

Without WebMCP, an agent would need a separate integration layer just to reach a memory service. Recall instead makes the memory itself a web capability surface: the agent discovers tools from the website, the tools operate against the website's state, the browser mediates each call, and the user decides which capabilities exist at all. That is a relationship that is difficult to reproduce cleanly with a conventional hidden memory API.

Challenges We Ran Into

Tool lifecycle management was one of the harder WebMCP problems. registerTool() throws a duplicate-name error if a tool is registered twice, which matters as soon as a user toggles a permission and the tool surface needs to be rebuilt. We solved this with an AbortController per registered tool and explicit cleanup before re-registration, so permission changes never collide with the existing surface.

Data layer constraints: Prisma's Rust query engine rejects libsql:// URLs outright, since its SQLite provider only accepts file: URLs. We worked around this by calling @libsql/client directly — the same client the Prisma adapter wraps internally — while keeping the Prisma schema as the source of truth for the data model and DDL generation.

Real-time updates in serverless: We use a small socket.io service for near-instant activity events, with a polling fallback if the WebSocket connection does not establish within a few seconds. That keeps the activity feed working even in environments such as Vercel's serverless functions, where a persistent socket service cannot run directly.

Accomplishments That We're Proud Of

The architectural inversion isn't just a pitch — it's a deployed, working product. The website is the memory, the agent is the consumer, the browser is the trust boundary, the capability token is the credential, and the audit log is the receipt, end to end, in production.

We're especially proud that user governance over agent capability actually works: disabling addFact narrows the capability token's scope, the agent is blocked from calling it, and re-enabling it restores access, verified on the live deployment rather than only in a local demo.

The audit trail is also independently checkable. Every tool call produces a signed, exportable event that anyone can verify against the included public key. That verification confirms that the signed event is authentic and has not been altered, providing evidence that does not depend solely on trusting Recall's database.

Just as importantly, Recall is implemented as a complete product rather than a protocol-only demonstration: a real dashboard, real search with a sensible fallback, real-time updates, and a settings page that directly changes the agent's available capabilities, all integrated into one deployed application.

What We Learned

The biggest lesson was architectural: user governance over agent capability is more compelling than "AI controlling a website." The interaction that matters isn't simply the agent calling a tool; it is the user disabling a capability and seeing the agent immediately blocked from using it.

That reframes WebMCP from "AI can operate websites" into "the website can define and enforce what an AI agent is allowed to do."

We also learned that restraint is a strength. We built an LLM-based semantic search layer, tested it, and removed it because the deterministic substring-and-tag approach was easier to verify, explain, and trust. We deliberately optimized the MVP for verifiability and user control rather than making the architecture dependent on opaque retrieval behavior. The innovation is where the agent boundary sits, not how many technologies get placed behind it.

WebCrypto also proved genuinely useful for this model. ECDSA P-256 signing in the browser, with the public key exported as a JWK for verification, provides a practical mechanism for producing verifiable receipts for agent-to-website interactions.

What's Next for Recall

Recall is a reference architecture for user-owned agent memory, not a single integration. ChatGPT is the first consumer, but the deeper goal is a memory boundary that can serve multiple agents without transferring ownership of the memory to any one agent platform.

The near-term roadmap includes direct ChatGPT OAuth once third-party OAuth ships to developers. Optional embedding-based semantic search is planned as a consent-gated enhancement layered on top of the deterministic query rather than a replacement for it. We are also building a mobile-first memory canvas for day-to-day fact management and a public verifier that lets anyone paste a JWS bundle and verify its signature without running a script.

Longer term, the goal is bigger than one ChatGPT integration. As agents take on more autonomous actions, persistent memory becomes more useful, and the consequences of opaque, agent-controlled memory become more significant. Recall's inversion — treating the website as the memory boundary rather than the agent platform — is a pattern that can apply to any conversational agent that needs persistent, user-trusted memory.

We want to publish Recall as a reference architecture other builders can adopt: the website is the memory, the user owns it, and the agent is only ever a consumer of what the user allows.

Built With

Share this project:

Updates

Submission history