Tenet

Inspiration

Every LLM agent with "memory" today is really RAG over a transcript log — append-and-retrieve. That's fine until a fact changes: you move cities, change jobs, update a preference — and the agent still has both the old and new statement in its retrieval pool, sometimes surfacing the stale one. On a controlled benchmark a strong RAG memory falls from 100% → 50% current-value accuracy as one fact is updated 2→12 times, and the collapse is identical under gpt-4o-mini and gpt-4o readers — so it is structural, not reader weakness. Track 1 names exactly this gap: facts that change, forgetting what's stale, recall under a small context budget.

Tenet is an original project built new during the Hackathon Submission Period — the engine, benchmarks, MCP server, adapters, and Alibaba Cloud deployment were all created for this entry.

What it does

Tenet reframes agent memory as a self-consistent belief state — a compact, supersession-aware set of current facts over plain SQLite + numpy (no graph DB, no vector service):

• Bi-temporal supersession. Every fact carries event time (valid_at/invalid_at) and transaction time (created_at/expired_at). A changed value is superseded — retired to history, not overwritten — so recall() returns the truth now and recall(as_of=t) time-travels to "what I believed in March."
• Write-time distillation. Qwen (qwen3.6-flash) turns raw messages into atomic subject::attribute facts, so later updates collide and supersede deterministically instead of piling up as near-duplicates.
• Timely forgetting. A salience-weighted decay sweep archives stale, low-value memories; pinned identity facts are never forgotten.
• Zero LLM calls on read. recall, doubts, time-travel, budget-fill and adaptive multi-hop navigate() are embeddings + closed-form math — ~11 ms, flat from 1k to 100k facts.
• MCP-native. learn / remember / recall / doubts / time_travel / forget_stale / memory_stats — any MCP client gets persistent, self-managing memory with no glue code.

How we built it

Built on Qwen Cloud end to end, three Qwen Cloud APIs each doing what it's best at, behind one fail-loud provider layer that swaps Qwen/OpenRouter/Ollama by env var:

• text-embedding-v4 for retrieval, qwen3.6-flash for write-time distillation, qwen3.7-plus for the assistant reader.
• A learned drift model (dynamics.py): a closed-form Gamma–Lomax survival model fit per key class from the ledger's own supersession history — no hardcoded half-lives.
• An RTX-trained LoRA distiller (tenet-distiller-1.5b-v2, Qwen2.5-1.5B) for a fully local air-gapped write path — 6/6 clean-churn supersessions, 0.0 fabrication, 0.775 key-consistency, beating the cloud reference's own 0.707 on held-out eval.
• Ships as a real product: CLI, HTTP API + belief-ledger web demo, LangGraph BaseStore + LlamaIndex adapters, MCP server, and a 2-page paper + full preprint.

Challenges we ran into

• We pre-registered two success gates so a negative result couldn't be reframed as a win — and both failed, honestly reported. Confidence-routed reader spend: an 84-config sweep found no setting saving tokens within 2pp of baseline (p_valid is a currency signal, orthogonal to the relevance errors that sink extractive routing) — confidence stays annotation-only, regression-tested. Multi-hop decomposition scored identically to baseline (25.0 vs 25.0): FactConsolidation multi-hop is reader-reasoning-bound, not retrieval-bound.
• Stale raw-turn leakage under real churn. Superseded raw turns slipped past the write-time filter and outvoted current facts; fixed with a read-time, key-scoped consistency check — churn half-life rose <2 → 8 (reported as a partial, honest fix, not a full close).

Accomplishments that we're proud of

• Numbers that reproduce, in a field where audits show they usually don't (Mem0's 93.4% LongMemEval reproduces at 73.8%; LoCoMo's key is 6.4% wrong). Every Tenet number carries a Wilson 95% CI, four flags ship default-OFF because we measured them as no-benefit, and we publicly falsified our own churn claim before fixing it.
• Beats published SOTA on MemoryAgentBench FactConsolidation: single-hop 97.0% 94.8, 98.3, multi-hop 45.8% 40.9, 50.6 — on a weaker local-7B backbone. Every result reproduces from one command.

What we learned

Temporal correctness for agent memory does not need a graph database or an LLM in the read path — a stable subject::attribute key plus bi-temporal supersession delivers Zep's job with pip install and $0 per read. And the field's real gap isn't a new mechanism, it's evaluation discipline.

What's next

Publish tenet-memory on PyPI; wire OSS-backed snapshot/restore into the Function Compute deploy for durable storage; close the churn half-life gap to flat-32.

Built With

• alibaba-cloud
• dashscope
• fastapi
• function-compute
• langgraph
• llamaindex
• lora
• mcp
• model-context-protocol
• numpy
• python
• qwen
• qwen3.6-flash
• qwen3.7-plus
• sentence-transformers
• sqlite
• text-embedding-v4