AI Employee — Agentic Memory on CockroachDB

AI agent that never forgets a follow-up: describe a symptom today, upload a lab report next week, ask about it next month — CockroachDB remembers exactly what was said and read, across sessions.

Comment

Inspiration

In Benin, as in much of Francophone Africa, the law exists but remains largely inaccessible to the ordinary citizen, scattered across hundreds of pages on a government website, often in scanned PDF format, with no search function or way to track an issue over time. The CockroachDB × AWS Hackathon's premise, memory as the foundation of agentic systems, not an afterthought, matched a problem worth actually solving: what does it look like when an AI agent's memory is grounded in real, substantial knowledge instead of just what a user typed in the same session?

What it does

Continuum ingests the full corpus of promulgated Benin laws (~1,600 texts, scraped from the government's official site) into a CockroachDB vector index, and keeps a persistent, thread-based memory of every conversation. Ask about a legal situation, labor law, business registration, a land dispute; and the agent retrieves the actual law that applies via vector search, cites it by number and title, and grounds its answer in the real text rather than general knowledge. Come back weeks later with a follow-up in a new thread, and it can still recall relevant context from any past conversation through semantic search, even across threads. To be clear: it's not legal advice; it's an information layer to find the right law faster, with a clear recommendation to confirm anything important with a qualified professional.

How we built it

  • CockroachDB Cloud (Serverless) is the memory layer end to end: user_context roots each identity, conversations (scoped by thread_id) holds turn-by-turn history per conversation thread, laws catalogs the ~1,600 ingested legal texts, and memory_embeddings stores VECTOR(1536) embeddings with a distributed vector index (vector_l2_ops); personal conversation memory and the global law corpus live in the same table, scoped by a nullable user_id (NULL = shared knowledge).
  • Two CockroachDB access paths, deliberately separate: direct SQL (psycopg2) for fast read/write and vector search, and the Managed MCP Server exposed as a tool Claude can call itself mid-reasoning (list_tables, get_table_schema, read-only select_query).
  • AWS Lambda (Docker image) runs the agent loop: build context → call Amazon Bedrock (Claude Sonnet 4.5 for reasoning + tool use, Titan Embeddings G1 for vectors) → handle the tool-use loop → persist the turn. Exposed through API Gateway (HTTP API).
  • Amazon Textract OCRs the law corpus: the large majority of the source PDFs on the government site turned out to be scanned images with no text layer at all; see Challenges below.
  • The cluster was provisioned and managed via the ccloud CLI.

Challenges we ran into

  • Most of the source law PDFs are scans, not text. We only discovered this after building the ingestion pipeline and testing it against the real site: pdffonts/pdfimages showed zero embedded fonts, pure JPEG/CCITT scan images. A naive pypdf extraction would have silently indexed an almost-empty knowledge base. We added an Amazon Textract OCR fallback (async job via S3, used automatically when pypdf finds no text layer), which correctly recovered clean, accented French legal text from ~90% of the corpus.
  • Ingestion throughput mattered at ~1,600 documents. A naive one-at-a-time pipeline projected to ~10+ hours. Decoupling the "download + OCR" concurrency (I/O-bound, cheap to parallelize widely) from a shared, bounded worker pool for Bedrock embedding calls (to avoid throttling) cut the full run to under an hour.
  • Lambda Function URLs silently returned 403 despite a correct resource policy, switched to API Gateway HTTP API instead.
  • CORS almost killed the demo twice: API Gateway forwards OPTIONS preflight straight to the Lambda on a $default route (curl never catches this, since it doesn't send preflights); and file:// origins get no CORS headers from API Gateway no matter what the Lambda returns.
  • API Gateway enforces a hard, non-configurable 30-second integration timeout. Sequential per-chunk embedding calls blew past it on a modest PDF; parallelizing them (ThreadPoolExecutor) cut a 40-chunk document from ~44s to ~13s.
  • Vector search alone isn't enough for "summarize this document." We redesigned uploads to match ChatGPT/Claude's attach-then-send pattern: a file is staged, not uploaded, until you hit send, then its extracted text is injected directly into that turn's prompt.

Accomplishments that we're proud of

Ask a real legal question and the agent cites the actual law, number, title, and quoted text — retrieved live via vector search over a corpus we built ourselves from a messy, mostly-scanned public data source. The "memory used" panel in the UI surfaces those citations directly, so the agentic memory isn't a backend implementation detail; it's visible and checkable.

What we learned

Persistent memory is only half the story, an agent that only remembers what a user typed is a nice demo, not a tool that solves anything. Grounding it in real, substantial, verifiable knowledge (here, ~1,600 actual laws) is what turns "the agent remembers" into "the agent is useful." We also learned that production readiness is mostly about boring infrastructure edges, hard platform timeouts, CORS quirks, and here, discovering a data quality problem (scanned PDFs) that only showed up once we tested against the real source instead of a clean sample.

What's next

Expanding the law corpus (regulations, decrees, case law) and the same architecture to other countries' legal texts, nothing about the pipeline is Benin-specific beyond the source URL. Also still on the list: the CockroachDB Agent Skills Repo for operational tasks beyond conversational memory.

Built With

Share this project:

Updates

Submission history