ContextForge — Persistent Memory for AI Agents

Persistent memory for AI agents. Store everything intelligently. Retrieve only what matters.

Comment

Inspiration

Every AI agent today has the same embarrassing flaw: it forgets you the second the session ends. Ask an assistant to remember your dietary preference, your meeting-time preference, or the fact that you're mid-fundraise as a startup founder, and by tomorrow it's gone — unless the developer re-sends the entire conversation history on every single request. That's expensive, it's slow, and it doesn't scale.

We wanted to build the piece of infrastructure that should already exist: a memory layer that sits between an AI agent and its context window, deciding what's worth remembering and what's worth retrieving — so agents get smarter over time instead of resetting every session. That's ContextForge: persistent memory for AI agents. Store everything intelligently. Retrieve only what matters.

Try it out: contextforge-one.vercel.app — log in with demo@contextforge.dev / contextforge-demo to see the seeded "Nova" AI-assistant scenario, or head straight to the Playground tab to store and retrieve memories live.

What it does

ContextForge is a managed memory API that any AI agent can call:

  • POST /api/v1/memories — hand it raw conversation content, and Gemini extracts the atomic, typed facts worth keeping (preferences, tasks, decisions, relationships, and more), each tagged with importance, confidence, and staleness.
  • POST /api/v1/context — hand it a new query, and it returns only the memories relevant to that specific task — semantically ranked by Gemini, conflict-checked so contradictory memories don't both make it into context, and packed into a token-budget-aware compact string.
  • Every memory is versioned. Nothing is silently overwritten — when a user's preference changes, the old memory is marked superseded with a full audit trail, not deleted.
  • A built-in Memory Health Agent autonomously audits a project's memory for duplicates, staleness, and low-value entries, and proposes cleanup — but never applies anything without explicit human approval.

The dashboard makes this tangible: an Overview with live token-savings metrics, a Memories browser with full provenance, an interactive Playground to store and retrieve memories live, and full API key and usage management for a real multi-tenant SaaS.

How we built it

Stack: Next.js 16 (App Router) + TypeScript end-to-end, Tailwind CSS + hand-built shadcn-style components, PostgreSQL with the pgvector extension for semantic search, Drizzle ORM for schema and migrations, Auth.js for session auth, and Stripe for billing.

Gemini is not bolted on — it's the reasoning core. A single GeminiMemoryService centralizes every place Gemini's judgment adds value:

  • Extraction — turning raw text into structured, typed memories with an importance/confidence/staleness estimate.
  • Conflict detection — deciding whether two memories about the same topic contradict each other, and if so, which one should win.
  • Consolidation — merging near-duplicate memories (e.g. six separate conversation snippets about the same ongoing project) into one clean summary.
  • Ranking — at retrieval time, re-scoring candidate memories by actual relevance to the current task, not just recency or importance.

Every one of those calls goes through Gemini with strict structured JSON output (validated with Zod), retried with backoff, and — critically — backed by a deterministic heuristic fallback if Gemini is unavailable. The app is fully functional with zero API key configured; it just degrades from Gemini-quality reasoning to keyword/similarity heuristics, and the UI honestly labels which one produced each result (source: gemini vs source: heuristic) rather than pretending nothing changed.

For storage, semantic candidate retrieval runs as a pgvector cosine-similarity search scoped to (project, user), then Gemini reranks the candidate pool by relevance to the actual query before anything gets packed into the final context string.

Challenges we ran into

  • Google deprecated text-embedding-004 mid-build. We'd built and tested against it, then hit a wall of 404s. We migrated to gemini-embedding-001 and discovered the installed SDK's embedding call didn't support the new model's dimension-control parameter at all — so we dropped down to a direct REST call to explicitly request 768-dimension output matching our vector column, rather than accepting the model's 3072-dimension default.
  • Auth.js on a self-hosted target. Login worked in dev but silently failed once deployed outside Vercel, because Auth.js can't statically verify the request host the way it can on Vercel — fixed with trustHost: true, which is required for Cloud Run/Docker-style deployments.
  • Graceful degradation had to be real, not decorative. It would have been easy to fake "works without an API key" by stubbing responses. Instead every Gemini call has a genuinely independent, deterministically-tested fallback path (Jaccard similarity for conflict/duplicate detection, keyword-overlap ranking, feature-hashing embeddings) that we unit test on its own merits.
  • Making the Health Agent demo honest. Our first seed data was already "clean," so the Health Agent had nothing to find. Rather than fake results, we seeded intentionally messy data — real near-duplicate memories, a real stale entry, a real low-confidence filler memory — so the analysis output is genuinely earned.

Accomplishments that we're proud of

  • A fully working store → understand → retrieve loop where every step's reasoning is explainable: every selected memory shows why it was chosen, every excluded one shows why it was excluded.
  • Real, measurable token economics: the dashboard shows actual tokens avoided and percent context reduction per retrieval, not a marketing estimate.
  • Zero placeholder logic in anything load-bearing — auth, tenant isolation, API key hashing, rate limiting, and the memory pipeline are all real, tested implementations.
  • 44 automated tests (including full multi-tenant isolation checks) plus an end-to-end Playwright happy path, all green, plus a clean production build.

What we learned

That the hardest part of "AI memory" isn't storage — it's judgment. Deciding what's worth keeping, what contradicts what, and what's actually relevant to this task is exactly the kind of fuzzy reasoning that's miserable to hand-code with keyword rules and genuinely well-suited to a language model — as long as you can validate its output, retry sanely, and fall back gracefully when it's unavailable. We also relearned that "works when the API key is missing" is a feature worth actually building and testing, not just claiming.

What's next for ContextForge

  • Multi-modal memory (images, documents) alongside text.
  • Usage-based Stripe metering beyond the current flat-tier checkout.
  • A scheduled/autonomous mode for the Memory Health Agent, so cleanup recommendations can queue up between human review sessions instead of only running on demand.
  • An official JS SDK to sit alongside the current Python SDK.

Built With

Share this project:

Updates