Inspiration
Every year, humans lose billions of dollars trusting a stated reason nobody actually checked — a vendor's bank details "changed," a caller claims to be who they say they are. That fraud already exists at scale. What's new is that AI agents are starting to make these decisions instead of people — and when you ask an agent why it did something, its answer might not match what actually happened. Not a bug: a measured, published property of how these models reason. This project ports a pattern we'd already proven in a different domain — a citation-verification gate for legal AI, checking that a quoted line is a literal substring of its source — into a domain where the stakes are money instead of words.
What it does
Provenance is a deterministic verification gate for AI agents making financial decisions. An agent gathers evidence, decides, and cites which evidence it used. A separate gate — zero LLM calls, pure lookup — checks each citation against two facts established before the agent ever ran: was it actually touched during the agent's traversal, and was it verified. Every decision resolves to one of four outcomes: grounded (cited, touched, verified), unverified (cited and touched, never confirmed), fabricated (cited, never touched at all), or unsupported (nothing cited). The gate never uses an LLM to grade an LLM — using one probabilistic system to check another just stacks the same uncertainty twice.
How we built it
The graph is Jac-native, not decorative: an Evidence node fires a real ability when a walker physically visits it — touched is a structural event, not a claim logged after the fact. The LLM fills a plain obj (DecisionOutput), never a node directly, then those fields get copied into a persisted Decision — keeping the AI-facing schema and the storage schema independently auditable. The gate walker is deliberately excluded from inheriting the ability that records a touch, so it's structurally incapable of contaminating the record it's checking — enforced by the type system, not discipline.
We stress-tested the thesis two independent ways beyond the core path: a second walker uses visit [...] by llm(intent=...) to choose its own investigation route — verified to genuinely select, not just reorder, skipping irrelevant evidence entirely — and the gate grades it identically correctly regardless of the path taken. Separately, we wired in live web-grounding via Firecrawl: a citation can point at a real URL, get scraped in real time, and get checked as a literal substring of the actual page content, not a pre-seeded flag.
Challenges we ran into
byLLM's structured obj output crashes inside its own schema builder on our installed version — a real defect, isolated by elimination (no sem, a trivial parameter, same-file types, a clean reinstall, all still failed identically) before concluding it wasn't our code. We shipped a fail-closed raw-string fallback that keeps by llm() real in the repo rather than dropping to a workaround outside Jac. Along the way: a nested walker's .reports silently leak into the outer walker's stream at the wrong array index, not just returning empty as the docs describe. A model told explicitly, in its own instructions, not to wrap output in a markdown fence did so in roughly one run out of three anyway — now stripped defensively. And live-testing our own web-grounding check against real, unscripted claims surfaced a genuine scope boundary: a true fact stated as a paraphrase instead of a quote correctly reads as unverified, because the check refuses to accept a close-enough match as equivalent to a real citation — the same discipline that makes the whole system trustworthy, just discovered the hard way.
Accomplishments that we're proud of
91.1% of the tracked codebase is Jac. The gate's "zero LLM calls" claim isn't asserted — grep -c "by llm" against every deterministic file returns 0, checkable in one command. And the core thesis was proven three separate, independent ways: fixed traversal, a walker with genuine agency over its own path, and live verification against the open web — all landing on identical, correct verdicts.
What we learned
That "just use a deterministic check" sounds obvious only in hindsight — the harder discipline is refusing every tempting shortcut that quietly reopens the same probabilistic hole one level deeper, including inside your own stress tests. And that the sharpest bugs — the schema crash, the report leak, the fencing, the paraphrase boundary — all showed up specifically when we stopped trusting the happy path and started testing against real, unscripted input.
What's next for Provenance
Re-attestation — verified isn't static; re-checking a source later and finding it's changed should be able to flip a previously-grounded citation, provenance with a clock. Retroactive replay — pointing the same, unmodified gate at a past decision instead of a live one, for after-the-fact audit, not just live blocking. And a narrow, purpose-built entailment classifier as one additional evidence source alongside exact-match — never a replacement for it — to catch true claims stated as paraphrase, without ever asking one general-purpose model to grade another's honesty.
Log in or sign up for Devpost to join the conversation.