MemoryIR

MemoryIR proves which memory actually caused an AI agent's decision — before it acts.

Comment

MemoryIR — Project Story (Devpost submission)

Elevator pitch

MemoryIR proves which memory actually caused an AI agent's decision — before it acts.

Inspiration

Persistent memory is the feature every AI vendor is racing to ship — ChatGPT remembers your projects, Copilot remembers your org's documents, personal agents remember your preferences across months, not minutes. It's also, we realized, an attack surface nobody had instrumented. Three real incidents made that concrete for us:

  • SpAIware (Johann Rehberger, 2024–2025). A single webpage with hidden instructions, summarized once by ChatGPT, could write an attacker-controlled note into the user's persistent memory. From then on, every future conversation — weeks later, on a different topic — silently exfiltrated the user's messages to the attacker. OpenAI's own triage initially classified it as a "safety" issue, not a security one, and the full fix for the memory-injection path didn't land until roughly four months after disclosure. That gap — between "we know it's dangerous" and "we can actually stop it" — is exactly the gap MemoryIR is built to close.
  • EchoLeak / CVE-2025-32711 (Aim Security, 2025). A crafted email, never opened, never clicked, was enough for Microsoft 365 Copilot to pull attacker instructions into its own retrieval context and exfiltrate chat logs, OneDrive files, SharePoint content, and Teams messages. CVSS 9.3. Zero user interaction. It didn't exploit a code bug — it exploited the fact that Copilot trusted whatever its RAG pipeline handed it.
  • MemGhost (arXiv 2607.05189, 2026). The most damning result for our thesis: an automated attacker that emails a personal agent once, gets it to write a false "fact" about the user into long-term memory, hides the write from the user-visible response, and then steers the agent's answers in later, unrelated sessions — 87.5% end-to-end success against a production-grade agent stack in the paper's own benchmark. The attack doesn't need to compromise the model or the app. It only needs the agent to trust its own memory.

Three different vendors, three different attack mechanics, one identical root cause: every one of these systems trusted that a memory being retrieved meant it was safe to act on. Nobody was asking "did this memory actually cause the decision, or did the agent just cite it?" — the exact distinction between correlation and causation that any first-year statistics course would insist on, and that none of these production AI systems were computing.

That gap is not free. IBM's Cost of a Data Breach Report 2026 puts the average breach at $4.99M, with AI-related incidents running measurably higher and 97% of organizations reporting no adequate AI access controls at the time they were breached. Once agents get real write access to production systems — sending emails, filing tickets, moving money, editing records — an unverified poisoned memory isn't a chatbot giving a wrong answer, it's a compromised system silently taking wrong actions, over and over, until someone notices. We wanted to build the thing none of the three incidents above had: a way to check not what an agent says it remembers, but what actually caused it to decide something — before that decision executes.

What it does

MemoryIR sits between "the agent retrieved a memory" and "the agent is allowed to act." Before a protected action executes, it:

  1. Retrieves the agent's relevant memories from CockroachDB via vector search.
  2. Records what the agent claims it used to make its decision.
  3. Runs counterfactual interventions — removes each retrieved memory one at a time and reruns the agent to see whether the decision actually changes.
  4. For memories that turn out to be causal, walks their derivation lineage back to the original ground-truth memory they were consolidated from.
  5. Blocks the action if that causal chain conflicts with a protected policy.
  6. Exposes an independent forensic console, backed by CockroachDB's Managed MCP server, so anyone can ask "why was this memory influential?" and get an answer computed outside the app's own code path.

The core distinction: citation is what an agent says it used. Causation is what actually moved the decision when you take it away. MemoryIR only trusts the second one.

How we built it

CockroachDB is the persistent memory layer, not an add-on — Distributed Vector Indexing handles semantic retrieval, and the same database is the transactional ledger for every trace, claim, memory edge, and intervention result. We deliberately didn't split memory across a vector store and a separate app database; one system does both jobs, and Managed MCP lets us query it a second, independent way for forensics.

AWS hosts the whole thing: one Lambda function serves both the FastAPI backend and the built React frontend, Amazon Bedrock powers the embeddings (Titan Text Embeddings V2) and the agent's decisions, and API Gateway is the public, rate-gated endpoint in front of it.

The frontend is React/TypeScript/Vite/Tailwind with a sidebar console layout — a live security dashboard, a protected-action review flow, a memory lineage graph (React Flow), a trace explorer, and the forensic investigator.

Challenges we ran into

A few of these only surfaced once we stopped testing against a scripted mock and started running the real pipeline against live infrastructure:

  • A live-model bug that only showed up under real Bedrock traffic. Our mock provider always returned clean, well-typed attribution data. The real model didn't: it sometimes returned "importance": "low" instead of a number, and occasionally referenced a truncated or hallucinated memory ID in its JSON response. Both crashed the pipeline with a raw KeyError/ValueError the first time we ran real traffic through it. We had to add defensive coercion and validation that our deterministic tests had never exercised.
  • An N+1 query problem that only mattered at scale. Our dashboard summary endpoint queried the database once per trace. Against an in-memory store that's free; against CockroachDB Cloud over the network, once we'd generated enough real trace history, a single dashboard load took over two minutes and timed out. We rewrote it around batched bulk queries (one query for all interventions, one for all retrievals, one for all claims, regardless of how many traces are being summarized) instead of one round trip per trace.
  • A connection-caching "fix" that made things worse. Our first attempt at fixing the above cached a single database connection at module scope to avoid TLS handshake overhead per query. That introduced a subtler bug: a connection idle for several minutes could die silently on CockroachDB Cloud's side and then hang instead of erroring on reuse. We reverted to a simple, correct per-call connection rather than keep a stateful long-lived one — the batched queries alone were enough to fix the real problem.
  • Multi-hop causal tracing, honestly reported. Our own 48-scenario evaluation suite caught this: the intervention engine correctly traces one derivation hop back to the true ground memory (24/24 in testing), but at two hops it currently finds the immediate derived parent and stops one hop short of the true root (0/12). Rather than paper over it, we measured it, reported it, and put it on the roadmap.

Accomplishments that we're proud of

  • Turning "prompt injection is dangerous" from an abstract claim into a concrete, blocked-action demo with a real causal chain behind it.
  • Building a real 48-scenario evaluation harness — not hand-typed numbers — that runs through the actual production QueryEngine/InterventionEngine/AttributionEngine, reproducible from the command line, and shipping it as both a CLI and a live dashboard page.
  • A measured result we can stand behind: trusting an agent's own citation gets the true causal memory right 25% of the time in our test suite. MemoryIR's ground-traced provenance gets it right 75% of the time — three times better, and verifiable by anyone who clones the repo.
  • Finding and fixing real production bugs (not hypothetical ones) by actually running the system against live Bedrock and CockroachDB traffic instead of stopping at mocked tests.

What we learned

Persistent memory changes the threat model for AI agents. A prompt injection can become durable state, and durable state can influence actions long after the attack itself is gone. Citation logs aren't enough, because they show correlation — "the agent retrieved memory X" — not interventional influence — "removing X changes what the agent does." We also learned, the hard way, that a system's real failure modes only show up once you stop trusting your own mocks: every serious bug we fixed this week came from running against live infrastructure, not from unit tests.

What's next

  • Extend ancestor ablation from one derivation hop to full recursive multi-hop recomputation, closing the gap our own eval suite measured.
  • Add policy/trust labels to memory sources, and fail closed when a protected action's causal path is unknown or untrusted.
  • Add trace export for auditors and memory-risk dashboards across agents and teams.

Why it matters

The incidents above weren't stopped by better prompts or better filters — SpAIware and EchoLeak both bypassed content-level defenses because the malicious payload never looked like an attack; it looked like a normal memory. Detection has to happen one layer down, at the level of does removing this memory change the outcome, which is exactly the causal question citation-based logging can't answer. That's the gap MemoryIR targets, and our own eval suite shows why it's worth closing: trusting an agent's self-reported citation gets the true causal memory right 25% of the time; MemoryIR's counterfactual, ground-traced provenance gets it right 75% of the time. On a system authorized to send emails, move money, or edit records on your behalf, that's the difference between catching a poisoned decision before it executes and finding out about it in a postmortem.

References

Built With

  • amazon-api-gateway
  • amazon-bedrock
  • amazon-titan-embeycopg
  • amazon-web-services
  • aws-lambda
  • boto3
  • cockrdex
  • cockroachdb-managed-mcp
  • fastapi
  • mangum
  • pgvector
  • pydantic
  • pytest
  • python
  • react
  • react-flow
  • tailwind-css
  • typescript
  • uvicorn
  • vite
Share this project:

Updates

Submission history