Entorhinal

A memory for your AI that notices when a fact stops being true and checks your question against it. On STALE, Nemotron 3 Super goes from 6/36 to 18/36 with it.

Comment

Inspiration

Personal AIs now remember our lives, but they remember like a pile of notes: every fact stays equally true forever. Tell one you bike to work every day, mention three weeks later that your leg is in a cast, and it still suggests a bike route. Nothing in the second message says "I no longer bike", so retrieval cannot connect them. The STALE benchmark (arXiv 2605.06527) measures exactly this failure, and its published table shows how hard it is: Claude Opus 4.6 scores 28.3, GPT-5.4 15.7, mem0 8.3 and Zep 6.0; only the authors' own memory system reaches 68.0. We wanted a personal memory that knows when something stopped being true.

What it does

Entorhinal is a long-term memory for a personal AI, built on NVIDIA Nemotron models served by Nebius Token Factory.

  • It remembers in dated facts. Each conversation becomes facts about the user, dated by their session.
  • It notices change when it arrives. For every new session, Nemotron 3 Super with reasoning decides which older facts it makes untrue, directly ("we moved to Abu Dhabi") or by consequence ("my leg is in a cast" ends the daily bike ride), and marks them stale, linked to the fact that changed them, with the date and the reason.
  • It checks your question before answering. It lists what the question assumes about you, checks each assumption against the memory, and corrects an outdated one first, citing the newer fact and its date.
  • It does not agree with you when you are wrong. On STALE's premise-resistance questions it refused 6 of 12 outdated premises; reading the whole history, Nemotron 3 Super refused none and Nemotron 3 Ultra one. The check is grounded: it can only flag a fact the memory holds as stale, and every correction shows its source fact and date.

The live demo opens on a real 50-session life from the benchmark, with the same question answered by Nemotron 3 Super reading the whole history ("Yes, that sounds like a solid plan", judged wrong) and by the same Super with Entorhinal ("You're actually working offline now, you cut your internet line on October 15", judged right). Every live step, model call, token count and cost streams into a pipeline view and a three.js scene that moves only on real events.

How we built it

  • Write path, once per session: Nemotron 3.5 Lightning extracts facts in JSON mode; every older fact on the same subjects is gathered with no cap; Nemotron 3 Super with reasoning lists only the old facts the session makes untrue, each with its cause and a reason; the store keeps them, marked stale, never deleted.
  • Read path, once per question: Super with reasoning marks each assumption SAFE, OUTDATED or UNRESOLVED; Super then answers, correcting an outdated assumption first.
  • One model client of about 15 lines on Token Factory's OpenAI-compatible API, with reasoning switched on only where judgement matters, retries, refusal of cut-off replies, per-call pricing from the public catalog, and a replay cache keyed by the SHA-256 of each request, so every number we report re-runs for $0.
  • The app is a Python standard-library server with one SQLite memory per visitor (deleted after 24 hours), a page that loads nothing from any other origin, and a daily spend cap kept in Render Key Value.
  • Quality: 200 tests on recorded and scripted replies, and 252 mutation tests, each a deliberate break of one guarded line that must turn a named test red, run in a fresh clone before every merge.
  • A Hermes Agent memory provider, so the same memory can sit behind a personal agent.

Challenges we ran into

  • The fact that changed was out of sight. Our first one-session-at-a-time version showed the change judge only the 40 newest related facts. On a pilot, the fact each scenario changed was older, so it scored 1 of 6. With no cap, the same scenarios scored 6 of 6.
  • A misspelt marker. Super with reasoning sometimes ends with FIN: or FINY: instead of FINAL:; 21 of 141 judge replies lost their decisions until the client accepted every spelling.
  • Reasoning budgets. Asking Ultra for a verdict on every old fact used 11,832 of its 12,000 output tokens on a 326-fact memory. Asking only for the facts made untrue, on Super, fixed it at a sixth of the price.
  • Paying for cut-off replies. A reply cut at its token cap is refused but still billed; we now count every one against the spend caps.
  • Free hosting sleeps. The demo's daily cap lived on a disk that a sleeping free instance loses; it now lives in a Key Value store.

Accomplishments that we're proud of

  • The same model, tripled: on 12 STALE scenarios none of our fixes were made on, Nemotron 3 Super answered 6 of 36 reading the whole history and 18 of 36 with the memory. The 550B Nemotron 3 Ultra reading the whole history also got 6 of 36, so a pipeline on the smaller models tripled the largest model's score. Judge: Nemotron 3 Ultra with STALE's own prompt. Premise resistance: 6 of 12 with the memory, 0 (Super) and 1 (Ultra) reading everything.
  • About $0.37 to build a 50-session memory and answer its three questions; one question costs one to two cents.
  • Every number re-runs for $0 from recorded replies, and the demo's recorded answers are re-derived from them and checked against the run before they are shown.
  • A demo that shows the difference in one screen, with every live model call's time, tokens and cost visible.

What we learned

  • Deciding what changed when the change arrives works far better than hoping a model notices it at question time among hundreds of facts.
  • Reasoning is worth paying for in exactly two places, judging change and checking premises; everywhere else a fast model without reasoning does the job.
  • Measure before claiming: a two-scenario pilot caught the cap defect that would have sunk the full run.

What's next for Entorhinal

  • Retry cut-off judge replies (52 of about 580 in the measured run), then re-measure on all 400 STALE scenarios.
  • Teach extraction to tell facts about the user from text they paste: on PrefEval at 300 turns the memory followed a stated preference 17 of 38 times, against 29 of 38 for plain retrieval.
  • Check the answer's own claims against the memory after it is written.
  • Run the Hermes provider inside a real Hermes Agent as an always-on personal assistant.

Built With

Share this project:

Updates

Submission history