Continuum

Institutional memory that never goes down — an SRE incident copilot built on CockroachDB as its agentic memory layer

Comment

Inspiration

AI agents are moving from experiments into production — writing code, running pipelines, and now handling incidents. But almost every "agent memory" stack we looked at was the same shape: a vector store bolted onto a database that was never designed for it, synced by hand, degrading under load, losing state on failover.

We picked the domain where that gap is least forgivable: incident response. An SRE copilot's whole job is to stay useful when everything else is on fire. If the agent's own memory can go down during exactly the kind of event it exists to help with, it has failed at the one thing that matters. That's the idea Continuum is built to prove out: a memory layer that is actually always-on, not just described that way — CockroachDB as a single distributed system of record for both the semantic knowledge (past incidents) and the transactional state (the incident being handled right now), with the agent able to check its own memory's health as a first-class reasoning tool.

What it does

Continuum is a chat copilot for on-call engineers:

  • An SRE describes symptoms in plain language.
  • The agent (Groq, real tool-calling — not a hardcoded pipeline) decides for itself whether to search the incident knowledge base, check the live health of its own database, both, or neither — based on what the conversation actually needs.
  • Semantic recall: CockroachDB's Distributed Vector Indexing ranks 18 seeded past incidents by cosine similarity against the live symptom description, and the agent cites the exact match inline ([INC-xxxx]), clickable in the UI to jump straight to the evidence.
  • Self-introspection: via CockroachDB's Managed MCP Server, the agent can query the live state of the cluster that stores its own memory — node state, version, active query load — when it suspects the incident is database-related. This isn't a static status badge; it's a tool the LLM chooses to invoke.
  • Durable state: every message, hypothesis, and citation is written transactionally to CockroachDB as it happens. Reload the page, or let the Lambda cold-start, and the incident's full timeline is still there.
  • A memory that grows: resolving an incident embeds its summary and folds it back into the knowledge base. The next similar incident will find this one too — the system gets smarter from the incidents it actually handles, not from a static seed set.

How we built it

CockroachDB tools (2+):

  • Distributed Vector Indexing — VECTOR(768) column + vector_cosine_ops index on the incidents table; embeddings from Gemini, retrieval-tuned (RETRIEVAL_QUERY vs RETRIEVAL_DOCUMENT task types).
  • Managed MCP Server — authenticated server-to-server via a scoped service-account API key (not the interactive OAuth flow), called directly from the Lambda agent as a live tool.
  • ccloud CLI — cluster provisioning and inspection during development.

AWS services:

  • AWS Lambda — runs the agent, Python 3.12, deployed from a Windows dev machine by cross-compiling dependencies for Lambda's Linux runtime (manylinux wheels, no Docker or AWS CLI required to build the package).
  • Amazon API Gateway — a single $default HTTP API route; the Lambda does its own request routing, so the entire deployment is one idempotent, rerunnable script.

Rest of the stack: Next.js 16 frontend on Vercel, Groq (openai/gpt-oss-120b) for tool-calling reasoning, Gemini for embeddings. Every piece of this runs at $0 — CockroachDB Serverless, Groq, and Gemini all have genuine no-card free tiers, and AWS Lambda/API Gateway stayed inside free-tier limits for this project's scale. That was a deliberate constraint, not an accident: it forced real architectural decisions (see below) instead of throwing paid infrastructure at problems.

Challenges we ran into

We're including these because they're evidence the system was actually built and tested against real services, not stubbed:

  • Cross-compiling for Lambda from Windows. psycopg2-binary and friends have native extensions; a plain local pip install would have bundled Windows binaries into a Linux Lambda package. Solved with pip install --platform manylinux2014_x86_64 ... — no Docker needed.
  • The official mcp Python SDK broke cross-platform packaging. It carries a Windows-only pywin32 dependency, and pip evaluates that marker against the host OS regardless of the --platform target — so it could never resolve cleanly when building on Windows for Lambda. We replaced it with a ~60-line direct JSON-RPC client over requests, which turned out simpler and more reliable anyway — a live check showed CockroachDB's MCP server is stateless per-request, so no SDK session machinery was ever needed.
  • The agent misread its own health data. CockroachDB Serverless reports node_count: 0 for every region by design (topology is abstracted away) — the agent initially interpreted that as "the database is down." Fixed by making the tool's return payload explain its own semantics, and verified the agent's answers changed accordingly.
  • CockroachDB Cloud Serverless doesn't expose node-level failure control — that's a Dedicated/Advanced-tier Cloud Console feature. To demonstrate the resilience mechanism honestly, we run a separate local 3-node cluster on the same CockroachDB version as production and document clearly, in both the README and the demo video, that this is a technical demonstration — not footage of the production cluster itself being killed.
  • GitHub Codespaces' Docker-in-Docker never actually attached, even after a full container rebuild — abandoned that path in favor of Docker Desktop locally, which worked once we found it installs to a per-user path rather than the usual Program Files.

Accomplishments that we're proud of

  • Everything is deployed and verified live, not just working on localhost: a real CockroachDB Serverless cluster, a real Lambda behind API Gateway, a real Vercel deployment — all tested end-to-end against each other, with CORS confirmed from the exact production origin.
  • A genuinely closed self-improving memory loop. Resolving an incident isn't a UI gesture — it measurably adds a new, correctly embedded row to the vector index, provably increasing what the agent can recall next time.
  • A captured, reproducible resilience proof: kill one of three CockroachDB nodes mid-workload, watch the other two keep serving the exact same data with zero interruption, restart it, watch it rejoin and recover automatically — output captured and documented, not just asserted.
  • Production-security-minded from the start: the AWS deploy runs under a scoped IAM user (only Lambda, API Gateway, and pass-role rights to one specific execution role — no admin access), and the MCP service account is scoped to Cluster Operator on a single cluster, not an org admin.

What we learned

That "agentic memory" is a genuinely different engineering problem from either a cache or a vector store alone — an agent needs the same system to be its fast semantic recall, its durable working state, and a introspectable target it can reason about, without the consistency gaps that come from stitching three separate systems together. CockroachDB's answer to that — pgvector-compatible vector indexing living in the same transactional database, plus an MCP surface the agent itself can call — is a materially different shape than "add a vector DB next to your app DB." We also learned exactly where CockroachDB Serverless's fully-managed abstraction trades away visibility (node-level control) in exchange for zero ops burden — a real, honest tradeoff worth understanding rather than glossing over.

What's next

  • Automatic severity classification and escalation instead of a fixed default.
  • Multi-region CockroachDB deployment to demonstrate genuine cross-region failover once the project graduates off the Serverless free tier.
  • Slack/PagerDuty webhook ingestion, so incidents open themselves instead of requiring manual description.
  • A lightweight admin view over the Knowledge Base panel (currently a visual placeholder) for curating/editing past incidents directly.

Built With

Share this project:

Updates