Engram

Give your AI agent a memory that works like a brain: capture fast while awake, then consolidate, forget, and reconcile during a sleep/REM cycle. A separable memory MCP service, built on Qwen.

Comment

Engram

Inspiration

Every AI agent today has amnesia. It's brilliant inside a single conversation and a blank slate the moment it ends. The industry's answer (stuff more tokens into a bigger context window) is expensive, doesn't scale, and still isn't memory: it's a bigger short-term buffer, not knowledge that persists, refines, and lets go.

Brains don't work that way. You don't consciously index your day. You live it fast, and then sleep does the filing. Neuroscience calls it memory consolidation: during slow-wave and REM sleep the hippocampus replays the day's episodes, the important ones harden into cortical knowledge, the trivia fades, and contradictions get reconciled. That single idea, capture cheaply while awake and do the expensive thinking while asleep, became Engram.

I wanted to build the memory layer I wished existed: a separable service any agent can plug into over MCP, that gets smarter overnight instead of just bigger.

What it does

Engram is a self-managing memory layer for AI agents, built end-to-end on Qwen. It splits memory the way a brain does:

  • Online path (hot, cheap, runs on every message): memory.write captures each episode, scores its importance without an LLM, embeds it (text-embedding-v3, 1024-dim), and dedups by content hash. memory.search does hybrid recall: vector + Postgres full-text keyword + graph Personalized-PageRank (HippoRAG-style multi-hop) → gte-rerank → a token budgeter that packs the winners under a fixed budget using relevance, recency, importance, and MMR diversity, and returns the full packing trace (every candidate and sub-score, kept or dropped).
  • Sleep / REM cycle (offline, the hero): while idle or on a schedule, seven checkpointed, cost-bounded steps run: forget the stale (demote to a cold tier, never delete), cluster survivors, consolidate each cluster into a durable semantic note (qwen-max), merge a knowledge graph, reconcile contradictions bi-temporally, synthesize new connections, and rewrite the user profile. Each cycle emits an observable report.

Forgetting is demotion, not deletion. A faded memory still reappears under explicit deep recall. A Qwen agent on Telegram / WhatsApp (via a vendored NanoClaw runtime, engine = Qwen Code) is the vehicle that shows it off, and a brain viewer visualizes the graph, the dream trace, a two-brains (memory vs. no-memory) comparison, and a live proof panel.

How I built it

A TypeScript pnpm monorepo, cleanly separated so the memory layer is the product and everything else is scaffolding around it:

  • packages/memory/: the hero, the MCP server (service.ts, online path) and the sleep/REM cycle (sleep.ts).
  • packages/shared/: infra plus all Qwen access behind one interface (qwen/): qwen-max (consolidation/synthesis/profile), qwen-turbo (extraction/contradiction judging), text-embedding-v3 (embeddings), gte-rerank (rerank), with a deterministic offline mock so the whole system runs with no API key.
  • packages/viewer/: read-only API plus React neural-graph UI.
  • packages/eval/: an 11-gate eval suite.
  • nanoclaw-v2/: the agent runtime. deploy/alibaba/: Alibaba Cloud config-swap deploy.

Storage: Postgres + pgvector (episodes, notes, entities, edges, profile), Redis/Tair for queues, blob/OSS for the encrypted cold archive. Because all infra sits behind packages/shared interfaces, deploying to Alibaba (AnalyticDB for PostgreSQL, Tair, OSS, Function Compute + EventBridge for the sleep schedule) is a single config swap: ENGRAM_INFRA=alibaba.

Some of the design choices I am happiest with:

  • Importance scoring with no LLM call on the hot path: a cheap heuristic (base 0.4, +personal pronouns, +durable markers like always/allergic/deadline, ±length), so writes stay fast; sleep later re-rates 1–10 with the model.
  • Never drop a write: if embedding fails, the episode is stored with a NULL vector and a reembed job goes to a dead-letter queue that a drain repairs later.
  • Personalized PageRank recall: with restart $\alpha = 0.5$ over the undirected entity graph,

$$r = \alpha \cdot s + (1-\alpha)\, W^{\top} r$$

seeded from the query's entities (≤50 power-iterations), aggregating node mass onto the notes that cite those entities, multi-hop association in one pass.

  • Decay you can reason about. Retention combines importance, age, and access:

$$\text{retained} = \text{importance} \cdot 0.5^{\,\text{age}/30\text{d}} \cdot \big(1 + 0.4\ln(1 + \text{access_count})\big)$$

Below threshold and not pinned or recently accessed → demoted, not destroyed.

Challenges I ran into

  • Making forgetting reversible. The hard part of a forget sweep isn't deleting, it's not deleting. I needed junk out of default recall and consolidation, yet reachable on demand. The cold-tier-plus-deep-recall design took several iterations to get right so "forgotten" never meant "gone."
  • Ordering the sleep cycle. Forget has to run first. Consolidating before pruning bakes trivia into durable notes permanently. Getting the seven steps in the right order, each checkpointed so a crash resumes mid-cycle, was subtle.
  • Cost-bounding an unbounded process. Consolidation and synthesis on qwen-max can run away. I added a per-tenant cent cap with clean early-stop so a cycle degrades gracefully instead of burning budget.
  • Blending four retrieval signals (vector, keyword, graph-PPR, core memory) into one ranked pack without any one drowning the others, hence min-max normalization, the 0.5·relevance + 0.2·recency + 0.2·importance + 0.1·diversity weighting, and MMR to kill near-duplicates.
  • Proving it works. Memory quality is fuzzy; I forced it into hard gates (recall, timely forgetting, limited-context recall, contradiction/update resolution, RAG, no-confabulation, latency).

Accomplishments that I'm proud of

  • An 11-gate eval that passes 3× on real Qwen, all green: recall, timely forgetting, limited-context recall, contradiction/update resolution, RAG retrieval + answer, no-confabulation, and ~200 ms p95 on the hot path.
  • A demo that tells the whole story in 60 seconds: an 11-act narrative on one persona: teach facts → ask both brains → answer from an uploaded PDF (and say "I don't know" when it can't) → change a fact → 💤 Dream → recall the new value → recall under a tight budget → show forgetting is demotion, not deletion.
  • A genuinely separable, 100%-Qwen memory MCP service that's not welded to the agent. The agent is just a vehicle.
  • Runs fully offline on a deterministic mock, so anyone can try it with zero setup and flip to real Qwen with two env vars.

What I learned

  • The bottleneck for agents isn't reasoning, it's memory management, and the neuroscience framing (fast capture, slow consolidation) is a genuinely good engineering blueprint, not just a metaphor.
  • Cheap-write / expensive-consolidate is the right economic split. Keeping LLM cognition off the hot path is what makes persistent memory affordable at scale.
  • Observability sells memory. Exposing the packing trace and the dream report turned an invisible, hand-wavy system into something you can watch think, and that's what made the demo land.
  • Interface-first infra pays off. Hiding Qwen and all storage behind interfaces gave us the offline mock, local-vs-cloud parity, and a one-line Alibaba deploy almost for free.

What's next for Engram

  • Multi-agent shared memory: one consolidated memory layer serving a team of agents, with per-tenant isolation and selective sharing.
  • Smarter sleep scheduling: trigger consolidation from cognitive load and novelty instead of a fixed timer, closer to real sleep pressure.
  • Richer knowledge graph: typed relations and temporal reasoning over the bi-temporal store ("what did I believe, and when?").
  • More channels & modalities: WeChat, voice, and image memories.
  • Memory portability: export/import your memory graph and take it between agents. Your memory should belong to you, not to whichever assistant happened to record it.

Built With

Share this project:

Updates