ContextOS — Persistent Memory for AI Agents

Upload once, extract once, remember forever, forget honestly. A decaying agentic memory layer on CockroachDB with vector + graph retrieval, powered by Amazon Bedrock.

Comment

ContextOS: A Persistent, Decaying Memory Layer for AI Agents

The Inspiration: Why AI Needs to Forget

Amnesia is the default state of modern AI. Every time a user starts a new session, the agent starts from zero. The industry standard solution, Retrieval-Augmented Generation (RAG), attempts to fix this by dumping raw text chunks into a vector database. But classic RAG is fundamentally flawed: it treats a trivial meeting note from three years ago with the exact same weight as a core architectural decision made yesterday. It never decays, it never strengthens, and it never connects.

Human memory does not work like a hard drive. Humans distill experiences into concepts, connect them relationally, reinforce them through use, and honestly forget what no longer matters. ContextOS was built to bring this biological reality to AI agents. We wanted to build a system where an agent processes a document exactly once, distills it into a living knowledge graph, and retrieves it forever—with memories that decay when neglected and strengthen when used.

How We Built It: The Architecture

ContextOS is not a traditional RAG pipeline. It is a multi-model memory engine powered entirely by CockroachDB and AWS.

The Role of CockroachDB (The Memory Substrate)

In most AI stacks, engineers are forced to stitch together three separate databases: Postgres for metadata, Pinecone/Milvus for vectors, and Neo4j for graphs. This creates synchronization nightmares, latency spikes, and operational overhead.

We used CockroachDB as the single, unified source of truth for all three paradigms:

  1. Relational Store: Manages user scoping, document metadata, session buffers, and processing job queues with strict serializable isolation.
  2. Distributed Vector Indexing: Stores 1024-dimensional Amazon Titan embeddings in a native VECTOR(1024) column. We leverage CockroachDB's distributed vector index for Approximate Nearest Neighbor (ANN) search, eliminating the need for an external vector DB.
  3. Graph Store: Implements a typed, confidence-weighted adjacency list via the relationships table, allowing the system to perform 2-hop graph traversals using standard SQL joins.

The AWS AI Layer

  • Amazon Bedrock (Nova Pro): Powers the "One-Pass" concept extraction during ingestion and intent classification during retrieval.
  • Amazon Bedrock (Titan Embeddings v2): Generates the 1024-dimensional vectors for semantic search.
  • Amazon S3: Acts as the encrypted cold-storage tier for raw document artifacts and forgotten memories.
  • AWS Lambda: Executes the asynchronous ingestion pipelines, scaling to zero when idle.

System Architecture Diagram

[ Documents / Chats ]
       │
       ▼
┌─────────────────────────────────────────────────────────┐
│  INGESTION LAYER (AWS Lambda + Bedrock Nova Pro)        │
│  Parse ➔ Structural Split ➔ 1-Pass AI Concept Extract  │
└──────────────────────────┬──────────────────────────────┘
                           │
       ┌───────────────────┼───────────────────┐
       ▼                   ▼                   ▼
[ Titan Embeddings ]  [ Bucket Store ]  [ Relationship Mapper ]
       │                   │                   │
       └───────────────────┼───────────────────┘
                           ▼
┌─────────────────────────────────────────────────────────┐
│             COCKROACHDB (The Memory Substrate)          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐   │
│  │  Relational  │  │    Vector    │  │    Graph     │   │
│  │   Metadata   │  │ VECTOR(1024) │  │  Adjacency   │   │
│  └──────────────┘  └──────────────┘  └──────────────┘   │
└──────────────────────────┬──────────────────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────┐
│  RETRIEVAL LAYER (Query Analyzer + Scorer + Assembler)  │
│  Vector (ANN)  +  Text (ILIKE)  +  Graph (2-Hop Join)   │
└──────────────────────────┬──────────────────────────────┘
                           │
                           ▼
                  [ AI Context Injection ]

The Memory Model (Mathematics of Forgetting)

Every concept extracted by ContextOS is stored as a "Memory Bucket" with a floating-point strength value ( S \in [0, 1] ).

1. Exponential Decay Strength decreases over time based on the concept's importance score ( I ). High-importance memories decay slowly, while low-importance memories fade rapidly. $$ S(t) = S_0 \cdot (1 - \lambda)^{\Delta t} $$ Where ( \lambda ) is the decay rate (e.g., ( 0.10 ) for high importance, ( 0.20 ) for low importance) and ( \Delta t ) is days since last access.

2. Access-Driven Reinforcement When a memory is retrieved and injected into an AI prompt, it is reinforced, creating a biological feedback loop: $$ S_{new} = \alpha S_{old} + \beta S_{access} $$ Where ( \alpha = 0.7 ) (retention weight) and ( \beta = 0.3 ) (access boost).

3. The Forgetting Budget To prevent context-window overload, the Assembler enforces a hard budget (default 20 memories) with a diversity constraint, ensuring the AI receives only the most relevant, strengthened concepts.

Challenges We Faced

1. The Multi-Model Retrieval Latency Problem

Running Vector, Text, and Graph searches sequentially would result in unacceptable latency for a chat interface. The Solution: We engineered the Retriever module to execute all three searches in parallel using Promise.allSettled. By leveraging CockroachDB's distributed vector index for the ANN search and optimized SQL ILIKE pattern matching for text, we merged and scored the candidates in under 200ms.

2. Graph Edge Typing and SQL Inference

Mapping a graph structure onto a relational database introduced severe type-casting bugs. Our relationship store initially failed when comparing UUIDs from the documents table against text-based metadata in JSONB columns (e.g., unsupported comparison operator: <uuid> = <string>). The Solution: We rewrote the correlation engine to use strict SQL casting ($1::uuid, metadata->>'sourceDocumentId'::text) and implemented a hybrid indexing strategy that resolves bucket IDs by both canonical string labels and strict UUID foreign keys.

3. The "One-Pass" Extraction Hallucination

Early versions of our pipeline chunked text arbitrarily, leading to fragmented concepts and massive API costs. When we switched to a "One-Pass" full-document extraction using Bedrock Nova Pro, the model began hallucinating generic structural labels like "Introduction" or "Summary". The Solution: We engineered a strict, multi-layered system prompt with "Structural Forbidden Rules" and "Label Quality Rules". We built a custom ConceptExtractor validator in TypeScript that programmatically rejects sentence-fragment labels, penalizes definitions that merely restate the label, and enforces strict noun-phrase constraints before the data ever touches the database.

4. Architectural Bloat and Dead Code

The initial backend scaffolding contained dozens of unreachable files, unused dependency injections, and duplicate correlation logic that caused silent failures during ingestion. The Solution: We wrote a custom dead-code audit script that analyzed the TypeScript AST and import graphs. We systematically archived 48 unreachable files, cleaned the dependency container, and consolidated the ingestion pipeline into a single, clean, asynchronous queue backed by CockroachDB's processing_jobs table.

What We Learned

  1. Consolidation beats fragmentation. Using CockroachDB for relational, vector, and graph workloads simultaneously eliminated an entire class of distributed systems bugs. We never had to worry about a vector DB and a relational DB falling out of sync.
  2. Forgetting is a feature. By implementing the decay engine and cold-storage tiering (moving memories with ( S < 0.10 ) to S3), we proved that an AI agent becomes more accurate over time because its context window isn't polluted with outdated, low-signal noise.
  3. Agentic memory requires structural intent. Raw text chunks are useless to an agent. Distilling a 100-page PDF into 40 typed, weighted, and connected conceptual nodes transforms the database from a passive storage bucket into an active reasoning partner.

ContextOS proves that the future of AI agents isn't just about larger context windows; it's about building systems that remember what matters, connect the dots, and forget honestly.

Share this project:

Updates