Flashbulb

• 

  Architecture

Inspiration

Every production memory layer I looked at (Mem0, Zep, Letta) spends an LLM call on the write path. Every turn, it asks a model "is this worth remembering, and does it change anything?" That is a token tax that scales with the length of the conversation, and it gets paid on the boring turns that say nothing new.

They also mostly do not forget. When a user's fact changes, because they moved, switched jobs, or changed their mind, the old value stays in the store and pollutes retrieval from then on. A memory that only accumulates slowly fills up with contradictions.

Neither pain is hypothetical. Mem0 makes multiple LLM calls per write and publishes its own playbook to "cut AI agent memory costs 3-4x," so the vendor treats the write tax as a real problem. The 2026 STALE benchmark (arXiv:2605.06527) finds that even the best frontier model notices a stored memory is no longer valid only 55.2% of the time, because agents act on preferences that changed months ago with full confidence.

I wanted a memory controller that makes the keep, forget, or doubt call locally, for zero LLM tokens, and stays accurate when the user's world drifts.

What it does

Flashbulb is a memory sidecar for a Qwen agent. On every observation it decides, with no LLM call, whether to store it, whether it supersedes something already known, and how much to trust it later. Two signals drive those calls, and neither costs a token.

Surprise comes from a Titans-style neural memory (2.8M parameters) trained at test time. The memory learns to associate keys with values online, and its surprise on a new observation $x$ tracks its associative loss $\ell(M, x)$: how badly the current memory predicts that item. A novel fact scores high, a duplicate scores low, and the memory absorbs each item with one gradient step:

$$M \leftarrow M - \eta \nabla \ell(M, x)$$

Contradiction comes from a tiny local NLI model (deberta-v3-xsmall, 22M). It catches the case surprise cannot: a knowledge-update like "I moved to Chicago" is low-surprise, because the topic is already familiar, yet it must be stored and it must supersede the stale fact it corrects.

The whole write decision is one rule. Store a new fact $x$ when its surprise $s$ clears an adaptive threshold $\tau$ (it is novel), or when it contradicts something already in memory $M$ (it is a knowledge-update):

$$\text{store}(x) = (s \ge \tau) \lor \text{contradicts}(x, M)$$

The contradiction check does not run against the whole store. A cheap cosine pre-filter first narrows it to same-slot candidates, the facts whose embedding similarity to $x$ is at least 0.62, so the NLI model only ever judges facts that are about the same thing. When the contradiction fires it does double duty: it admits the correction and decays the record it contradicts, in a single write.

The agent answers with Qwen. It drives recall through Qwen's native function-calling: qwen-plus decides when to call a recall_memory tool, writes the query itself, and answers over the returned memories. Those come back tagged with confidence, so the agent hedges on stale facts and abstains ("I don't have that yet") instead of making something up. Qwen orchestrates the recall; the token-free gate still owns what gets kept and forgotten.

For Track 1, the payoff is something you can watch. Teach the agent facts in the dashboard, snapshot the memory to Alibaba Cloud OSS, close the session, reopen it, and the agent still remembers. The memory crosses the session boundary through the cloud, live.

How I built it

• The memory core (memory/sidecar.py) imports no LLM, so it stays provider-free by construction. Titans NeuralMemory produces the surprise signal, bge-small-en-v1.5 embeddings plus SQLite hold the verbatim store, and deberta-v3-xsmall produces the contradiction signal. Zero LLM tokens touch the write path.
• Qwen (DashScope, OpenAI-compatible endpoint) runs in four roles: agentic recall through native function-calling (qwen-plus deciding when and what to recall), the agent's answerer (qwen-plus), the llm_extract baseline the gate eliminates, and the benchmark judge (qwen-max). One shim (llm_client.py) sits in front of the provider, and the write path never calls it.
• Alibaba Cloud hosts the FastAPI backend on an ECS instance (Ubuntu, Singapore) under systemd, and every memory snapshot persists to OSS through the oss2 SDK (cloud/oss_store.py). That file is the "runs on Alibaba Cloud services" proof and the engine behind the session boundary.
• The dashboard is an instrument panel: the surprise signal firing per turn, retention decaying as facts get superseded, a write-token counter (zero for the gate against a climbing baseline), and the Snapshot and Restore buttons that cross the session boundary.

The measured result

I built a pre-registered ablation to settle one question: does a drift-adaptive write policy actually beat static ones? Five write policies, the same embedder and the same top-k retrieval for all of them, so the only thing that varies is the write and forget policy. Any gap is that policy's doing.

write policy	clean recall (higher is better)	stale contamination (lower is better)
store-all	0.43	0.52
cosine dedup	0.43	0.52
SAGE-vMF (published SOTA density gate)	0.46	0.46
cosine forgetting	0.63	0.37
Flashbulb (ours)	0.82	0.06

Over 400 held-out drifting streams (a disjoint seed range from the one where I tuned the single threshold), Flashbulb beats SAGE-vMF by 0.35 on clean recall and cuts stale contamination by about 9x, still at zero write tokens. Every 95% bootstrap confidence interval on the paired gap excludes zero, win-rate runs from 0.85 to 0.99, and all three pre-registered bars pass.

What I learned

The core lesson was about signals, and it cost me a failed benchmark to see it. Surprise alone cannot tell a redundant duplicate from a knowledge-update, because both are low-surprise: the topic is already familiar either way. No amount of threshold tuning fixes that, because the two cases are genuinely indistinguishable on the surprise axis. You need a second signal that measures a different thing. Contradiction is that signal, and once it went in the design clicked, because it not only admits the update, it tells you which old record to forget. That is also why density gates like SAGE and cosine fail on updates: they read an update as a near-duplicate and drop it.

The second lesson was that a cheap classical signal can rescue an expensive learned one. The small NLI model is powerful but trigger-happy, so on its own it was unusable. A three-line cosine filter in front of it, deciding "are these two facts even about the same thing," is what made it trustworthy. The dumb signal guards the smart one.

The third was methodological. My first "passing" number was a tie I had half talked myself into, and pre-registering the bar plus holding out a disjoint seed range is what kept me honest when I went back to check. I trust the 0.35 because I wrote down what would count as a win before I ran it on data I had never tuned on.

Challenges I ran into

HuggingFace is unreachable from Alibaba's Singapore region, so every model download hung the first request indefinitely. I pre-cached the models locally, shipped them to the box, and ran fully offline with HF_HUB_OFFLINE=1.

The honest one: my first rigorous benchmark showed the method tying a naive baseline. Debugging that tie is what produced the insight above. A cosine pre-filter set too tight was hiding true contradictions from the NLI model, and surprise-only admission was silently dropping every knowledge-update. Loosening the same-slot gate and adding admission-via-contradiction is what turned the tie into a 0.35 win.

The NLI model also over-fires on parallel sentence structure. "Switched to a CrossFit gym" reads as a contradiction of "switched careers to PM," because the shape of the sentence matches. The same-topic cosine gate, tuned on a dev split and validated held-out, is what contains it.

Honest limits

• The drift benchmark is synthetic: controlled, labelled slot-updates. It isolates the write policy cleanly, but it is not natural dialogue.
• On LongMemEval's oracle knowledge-update slice (72 questions), forgetting hurts: store-all scores 0.69, flashbulb 0.33, naive cosine forgetting 0.18. In the oracle setting the answer model sees every evidence turn and resolves the update itself, so dropping the old value only removes context. Two honest notes go with that: flashbulb's precise forgetting nearly doubles naive forgetting, and it matches store-all on abstention (0.83). The slice is brutal for everyone; a GRPO-trained approach (Supersede, arXiv:2606.27472) reaches only 16.7% supersession accuracy on it. Forgetting earns its keep under distractor load, which the drift benchmark isolates and where flashbulb wins.
• Contradiction detection leans on a 22M NLI model reading same-slot swaps as contradictions. A swap phrased with no logical conflict falls back to retrieval recency.

Lineage disclosure

Flashbulb was first built for a prior hackathon (AMD). This is a significant extension for Qwen Cloud, and everything new was built during this submission period: the Qwen and DashScope integration, the Alibaba ECS and OSS deployment, the two-signal controller (admission-via-contradiction plus NLI forgetting), agent-visible confidence and abstention, the visible cross-session boundary, and the entire pre-registered drift benchmark with its 0.35 result. The repo is MIT-licensed and states all of this openly.

What's next

• Run the drift benchmark against real long-horizon dialogue under distractor load, where forgetting should help most.
• Expose the surprise, retention, and drift-state signals as first-class MCP resources.
• Swap in a stronger local NLI model to close the cross-topic false-positive gap.

Built With

• alibaba-cloud
• alibaba-cloud-ecs
• alibaba-cloud-oss
• bge-small-en-v1.5
• dashscope
• deberta-v3-xsmall
• fastapi
• function-calling
• html
• huggingface-transformers
• javascript
• mcp
• nli
• python
• pytorch
• qwen
• qwen-max
• qwen-plus
• sentence-transformers
• sqlite
• test-time-training
• titans-pytorch
• uvicorn