StaleCatch

• 

  The reveal: the funder moved the deadline from Nov 19 to Nov 9. Sam, told Nov 19 on Oct 6, rises to the top with a correction draft.

• 

  Before the check: 2 sources, 6 items built from them, 3 people already told. Every item records which source it came from.

• 

  Each source keeps its fetch time and the verbatim excerpt. The change is shown side by side; the calendar and plan are updated.

• 

  Approving the correction: StaleCatch drafts, you send. Nothing goes out automatically.

• 

  Cost receipt from Token Factory usage fields: Nemotron Nano scans for $0.0001, Nemotron Ultra reviews only what changed.

• 

  Proof panel: 18 real deadline changes from Internet Archive snapshots. 17 found; false alarms 8 of 10 with Nano alone, 1 after Ultra.

• 

  Architecture: Hermes on NVIDIA NemoClaw, an MCP memory server, Tavily re-fetch, Nemotron Nano and Ultra on Nebius Token Factory.

• 

  One real run as a Nebius Serverless Job: COMPLETED, 3 items re-derived, $0.0167 in Nemotron calls. Log committed to the repo.

• 

  The funder page in the demo is ours and labeled as a demo page, not a real funder.

Inspiration

Your assistant told your client the grant is due November 19. Last night the funder moved it to November 9. Your assistant still believes November 19, your client is gathering documents for it, and nothing will tell you until someone else does.

Blood banks solved the same problem decades ago. When a donation turns out bad, they trace every unit made from it and recall them, including the ones already given to patients. That procedure is called look-back (EU Commission Directive 2005/61/EC). Most assistants we have used remember well, but we could not find one that tells you which of its answers rest on a page that changed last night, or what you already sent because of it.

What it does

StaleCatch gives every fact your assistant holds a source record: URL, fetch time, and the verbatim excerpt. Every item built from it (a calendar entry, a plan, a message you sent) records its parents.

A small model checks the sources. When one changes, the items built from it go on hold, a large model updates only those items, and the message you already sent rises to the top of "Who's been told" with a correction drafted for you to approve. Nothing is sent automatically.

In our demo, a grant writer's client (Sam) was told November 19 on October 6. The funder's page, our own labeled demo page, now says November 9. One click runs the check: Sam's message rises from row 3 to row 1 with the old and new text side by side and a correction draft. On real models the whole reveal takes about 10 seconds.

The demo funder page is ours and is labeled as ours. Detection quality is measured separately on real page changes (see "Does it find real changes?").

How we built it

Hermes agent (NVIDIA NemoClaw) --MCP--> Memory server (FastAPI + SQLite: sources, items, parent edges)
Check (runs as a Nebius Serverless Job) --> re-fetch (Tavily / HTTP) --> Nemotron Nano compares --> Nemotron Ultra confirms + re-derives
Board: Sources | Built from them | Who's been told, cost receipt, bench panel

Text version of the same pipeline: Hermes on NemoClaw saves sources through an MCP memory server; a nightly check re-fetches each source, Nemotron 3 Nano compares old and new, Nemotron 3 Ultra confirms every alarm and re-derives only the held items; the board shows the result.

Why Personal AI: the assistant keeps persistent memory with a source for every fact, runs its checks as a background job, and only drafts corrections for the user to approve. The memory lives in the user's own SQLite file; only excerpts of the pages the user registers leave it, for the model calls.

Sponsor use at a glance

• Nebius: Token Factory (Nano and Ultra calls with logged usage), Serverless Jobs (one real run). Serverless Endpoints not used.
• NVIDIA: Nemotron 3 Nano and Ultra; NemoClaw with Hermes as the agent shell.

• Tavily: Extract re-fetches public pages only (verified on a public arts.gov page); our own demo page uses plain HTTP.

• Two model tiers, one receipt. Nemotron 3 Nano (nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B) watches every source. Nemotron 3 Ultra (nvidia/Nemotron-3-Ultra-550b-a55b) double-checks every Nano alarm and re-derives only the held items. Both run on Nebius Token Factory behind one gateway module. Every call logs the API's own usage fields; the cost receipt multiplies them by list prices and shows what "Ultra checks everything" would have cost. On the bench below: $0.17 as built, $0.34 with Ultra on everything.

• Enforced in code, not in a prompt. The memory server rejects derive without parents. A date guard checks every re-derived date: it must fall after today and on or before the new deadline; otherwise it retries once and marks the run incomplete rather than storing a wrong date.

• Nebius Serverless Jobs. The nightly check is a container job (Dockerfile, python -m donor_lookback.job). It ran once as a Serverless Job on 2026-10-26: job stalecatch-nightly-1, COMPLETED, 2 sources, 1 changed, 3 items re-derived, 0 errors, $0.0167 in Token Factory calls (Ultra on everything: $0.0182). The job seeded and served its own copy of the demo pages inside the container. No schedule is set up; we do not claim it has been running nightly.

• Hermes on NVIDIA NemoClaw. The agent saves sources and derives items through the memory MCP server. In five scripted runs it produced one source plus three items with correct parents each time (19 to 27 s per run), once we set NEMOCLAW_TOOL_DISCLOSURE=direct. Setup notes: docs/hermes.md.

• Tavily. Extract re-fetches public source pages; we verified it on a public arts.gov page. Our own demo page is fetched by plain HTTP (Tavily cannot reach it), and the fetch path is recorded on every source. In our runs Tavily did not make the decision on camera; we say so rather than imply it.

• Judge mode. https://stalecatch.vercel.app runs the board read-only with the seed, a real recorded run (replay) and the proof panel. It holds no Token Factory key. Live checks run locally.

Does it find real changes? (the archive bench)

python -m bench.run_bench replays real before/after page pairs from the Internet Archive. All 64 snapshots are committed, with both snapshot URLs per pair.

The 18 changes are mostly NIH Guide notices that move one funding opportunity's due date; each notice is dated, so anyone can check the pair. The one miss, the one false alarm (a rotating listing page), three controls the models could not compare, and one unverified pair excluded from the headline are all listed in the README and the board's proof panel. Run-to-run variance is large (Nano + Ultra found 16 to 18 of 19 across runs); read the table as one sample.

Challenges we ran into (what did not work)

• Nano alone cries wolf. It flagged 8 of 10 unchanged pages. The second look by Ultra is the reason the tiers exist: it cut that to 1.
• Reasoning mode made the watcher slow. Nano took about 16 s per call with reasoning on and 1.7 s with it off (chat_template_kwargs.enable_thinking=false).
• An early bench overstated itself. It printed "0 false alarms on 13 controls" while only 10 controls were ever compared. We fixed the report and said so in the README. A first control pool also hid a real change; we redrew it and describe each redraw.
• Archive curation is slow. The Internet Archive refused connections after about 60 requests, so we slowed to one index call per page and 15 s between snapshots. Real mid-cycle deadline changes are rare in archived pages (about 1 in 10 sampled).
• Hermes on NemoClaw needed three non-obvious settings: direct tool disclosure (the default dropped a save in one of three runs), HTTPS with a CA that has basicConstraints=CA:TRUE for the MCP endpoint, and a fresh sandbox instead of rebuild, which stripped the in-sandbox model key.
• No Hermes transcript was saved, so our evidence for that step is the setup notes and the five documented runs, not a log.

Accomplishments that we're proud of

• A recall that computes live from stored parent edges, including what already went out.
• A measured two-tier split with a receipt from API usage fields, not a slogan.
• A bench of 32 real archived pages (64 snapshots) that shows its misses first.
• A real Serverless Job run with job id, image and log committed to the repo.
• 127 tests pass offline; 38 commits since 2026-10-25, one author per commit, history kept as it happened.

What we learned

Make the cheap model do the watching and the expensive one do the doubting. Gate wrong answers in code (parent enforcement, the date guard), because a prompt will eventually let one through. Report the false alarms next to the hits; the second tier is only convincing when the first tier's mistakes are on the page.

What's next

A real nightly schedule for the Job; real Gmail and Calendar connectors instead of the internal outbox; a measured base rate of how often funder deadlines move; a sourced practitioner dossier.

Built With

• fastapi
• hermes-agent
• internet-archive
• model-context-protocol
• nebius-serverless-jobs
• nebius-token-factory
• nvidia-nemoclaw
• nvidia-nemotron
• python
• sqlite
• tavily
• vercel