Inspiration
Personal AI becomes more useful as it remembers us.
It can remember our preferences, habits, past decisions, and instructions across sessions. That persistent memory is what makes an assistant feel personal — but it also creates a new kind of risk.
What happens when the AI remembers something incorrectly, over-generalizes an old preference, or gives one memory too much influence?
A traditional activity log can tell us which memories were retrieved. It cannot answer the more important question:
If this memory had not existed, would the AI have made the same decision?
That question inspired RecallGuard.
RecallGuard is a counterfactual memory replay system for Personal AI. Its goal is not simply to show what an agent remembered, but to determine which memory actually changed its behavior.
What it does
RecallGuard captures the exact memory context behind a Personal AI decision.
For example, imagine an assistant remembers:
- The user usually prefers cheaper flights.
- The user dislikes overnight layovers.
- Economy class is fine.
Later, the user asks:
“Book me a flight to Tokyo next Friday.”
The assistant chooses the cheapest flight — but that option includes an overnight layover.
The user reports the decision as an incident.
RecallGuard then performs two types of replay.
First, it replays the original decision using the same request, available options, and memory context.
Then it performs counterfactual replays, removing one memory at a time:
All memories
→ Flight A
Without “prefer cheaper flights”
→ Flight B
Without “avoid overnight layovers”
→ Flight A
Without “economy is fine”
→ Flight A
If removing one memory consistently changes the selected action, RecallGuard marks that memory as having high decision influence.
The user can then quarantine that memory so it will no longer affect future Personal AI decisions.
The complete loop is:
Remember
→ Act
→ Unexpected Decision
→ Replay
→ Remove One Memory
→ Replay Again
→ Compare
→ Quarantine
How we built it
RecallGuard is designed as a pure-software Personal AI application.
The core stack includes:
- Nebius Token Factory for model inference
- NVIDIA Nemotron for original and counterfactual decision replay
- FastAPI + Python for the backend and replay engine
- SQLAlchemy + SQLite for persistent memory, decisions, incidents, and replay history
- Next.js + TypeScript for the interactive web interface
- Playwright for automated end-to-end testing and reproducible demo recording
The architecture separates model reasoning from counterfactual analysis:
Personal AI Request
↓
Persistent Memory
↓
NVIDIA Nemotron via Nebius Token Factory
↓
Original Decision
↓
Incident Snapshot
↓
Counterfactual Memory Variants
↓
Nemotron Replay
↓
Action Comparison
↓
Memory Influence
↓
Human Review / Quarantine
Nemotron is not used only to generate an explanation.
It performs the actual decision replay for both the original memory set and the counterfactual memory sets.
The application code constructs the replay conditions, validates structured model output, compares actions, measures replay consistency, and manages memory state.
This distinction is important to us:
The model makes the decision. RecallGuard tests what happens when the memory context changes.
Challenges
One of the biggest challenges is that LLM decisions are not perfectly deterministic.
If an agent produces a different answer every time, we cannot confidently say that removing a memory caused the behavioral change.
To address this, RecallGuard uses repeated replay runs and measures agreement across them.
For example:
Without Memory M001
Run 1 → Flight B
Run 2 → Flight B
Run 3 → Flight B
A memory that changes the action consistently provides much stronger evidence than a single replay.
Another challenge is distinguishing memory retrieval from memory influence.
A model may receive a memory in its context without actually using that memory to determine the final action. This is why RecallGuard does not rely only on the model saying “I used this memory.”
Instead, it tests the decision again without that memory.
We also need to preserve the exact original decision context — including the request, available options, memory IDs, model, and prompt version — so that an incident can be replayed meaningfully later.
What we learned
The most important insight behind RecallGuard is:
Memory retrieval is not the same as memory influence.
Knowing that an AI retrieved a preference does not prove that the preference changed its behavior.
Counterfactual replay gives us a stronger test:
Remove the memory and ask the same AI to decide again.
If the outcome changes consistently, we have evidence that the memory materially influenced the decision.
We also learned that Personal AI memory needs more than storage and retrieval.
As assistants become more persistent and autonomous, users will need tools to:
- inspect remembered information,
- understand how memories affect decisions,
- identify problematic memories,
- and regain control when those memories produce unwanted behavior.
Why it matters
Persistent memory is becoming one of the defining features of Personal AI.
An assistant that remembers nothing cannot become deeply personalized.
But an assistant that remembers everything without accountability creates a different problem.
A stale preference, misunderstood instruction, or over-generalized memory could silently influence decisions across many future sessions.
RecallGuard introduces a human-control layer between memory and long-term autonomous behavior.
Instead of asking only:
“What does my AI remember?”
we want users to be able to ask:
“Which memory changed what my AI decided to do?”
What's next
The first version of RecallGuard focuses on a controlled Personal AI decision scenario and single-memory counterfactual replay.
Future versions could extend this to:
- memory editing instead of only quarantine,
- memory expiration and confidence levels,
- conflicting-memory detection,
- multi-step agent actions,
- multiple Personal AI domains such as travel, shopping, scheduling, and communication,
- richer counterfactual experiments involving combinations of memories,
- and reusable memory-safety hooks for other Personal AI systems.
Our long-term goal is simple:
Personal AI should not only remember us. It should let us understand and control how those memories change its behavior.
Log in or sign up for Devpost to join the conversation.