A seller is about to publish a listing. The title says one thing, the photo shows another, the shipping promise contradicts the policy page, and the rating badge came from somewhere nobody can name. On the other side of the same page, a buyer is deciding whether to trust it. Claim Court is the check that runs at that moment: every claim on the listing is reconciled against its evidence by a deterministic gate — and the gate can refuse.

Claim Court is the commerce surface of why-agent, our deterministic verification layer. Claims arrive as evidence rows with citations, grouped by aspect (state, outcome, cause). Nemotron reviewer lanes on Nebius Token Factory advise; a deterministic reconciler decides, returning one of four verdicts: CORROBORATED, PARTIAL, CONFLICTED, or NO EVIDENCE. Models contribute claims, never verdicts — that boundary is the security model, because any integration can be poisoned.

We pointed it at our own pilot first. The recorded corpus leads with a real business — Shenail Hair Salon & Nail Bar in Niagara Falls, the SiteBefore pilot client (our family's salon) — using only its own public pages and the directories that list it, fetched 2026-10-30. The conflicts are real, and nobody edited them into existence:

  • Three aggregators rate the same salon 4.1 (Google, 28 reviews), 4.5 (Birdeye, 251 reviews, profile unclaimed), and 4.7 (BestProsInTown, 110 reviews). The gate returns CONFLICTED.
  • The salon's own site lists one phone number; its Fresha booking page lists a different one. CONFLICTED.
  • Yellow Pages publishes longer hours than the salon's own site and Google show. CONFLICTED.
  • Where the sources genuinely agree — the street address — the gate says so, and returns the honest PARTIAL it gives when no source speaks to cause.

Alongside the 7 recorded items sit 11 synthetic fixtures modelled on real listing patterns — a "100% handmade" mug whose supplier spec and photo read say machine-printed, a was-price anchored at a price the item never sold at, a "genuine leather" listing whose spec says PU. Every fixture is labelled synthetic in the data, in its title, and in the UI. No fixture names a real third-party seller: the court points at our own properties and invented listings, never at a stranger's shop.

The verdicts, computed by the reconciler (not arranged): 12 CONFLICTED, 5 PARTIAL, 1 CORROBORATED across the 18 claims. The one corroborated item is a deliberately clean synthetic listing — the gate can say yes, and shows its work when it does. Full corpus, sources, and aspect mapping: claimcourt/ in the repo; every verdict re-derives from the committed rows via claimcourt/score.py, which fails on drift.

The Courtroom: Nemotron Court. On the 14-case adversarial benchmark, the reviewer tiers don't just vote — they are audited. A deterministic citation audit takes each recorded lane verdict and lists the evidence it failed to account for. It flags the fast tier's false approval of a card that claimed a failed deploy had shipped (two unanswered conflict citations) and clears the deep tier on the same item. Across the corpus, the tiers and the gate disagree on 8 of 14 items; every disagreement is preserved in the transcript, never averaged. A live cross-examination round — each tier stating a position, citing evidence, rebutting the others under a strict protocol — is built and budget-capped; it is labelled NOT RUN in the demo, because it had not run when this was written, and we do not dress up un-run work.

Measured results (14-case adversarial benchmark, recorded live Token Factory run, 2026-10-28): the deterministic gate scored 14/14 with zero false approvals. The Nemotron lanes on the same corpus: Ultra 9/14 with one false approval, Nano 6/14 with two — including believing the card that claimed the failed deploy had shipped. The gate caught every trap the lanes missed. Reviewer-lane cost for the benchmark run: $0.033. A separate first 4-item live batch cost $0.0071 total. Full results, misses named: docs/BENCHMARK_RESULTS_2026-10-28.md in the repo.

How it works (all models on Nebius Token Factory):

  • Reviewer lanes — Nemotron 3 Nano (fast) and Nemotron 3 Ultra (deep). Each independently judges the claims and appends its verdict as a cited evidence row. The gate decides; disagreements are logged, never averaged.
  • Narration — Nemotron 3 Super (120B). Writes the explanation of each verdict; a two-pass checker verifies every value in the narration against the evidence rows. A value that does not literally appear in the evidence fails loudly instead of reaching you polished. In our first live run the checker caught exactly that.
  • Vision lane — Nemotron 3.5 Super VL. Reads listing photos and report screenshots, extracting checkable claims (materials, amounts, status) that join the evidence rows.
  • Live-evidence lane — Tavily. A claim about the outside world is checked against live pages, and a page counts only if the claimed value literally appears in its content. Tavily's own generated answer is never treated as evidence — it is another model's assertion, which is the thing being checked. On 30 October 2026 we ran the Tavily lane live in an on/off ablation over n=4 of our own planning claims: adding web evidence changed the verdict on all four — each moved from PARTIAL to CONFLICTED — because the lane credits a page only when the claimed value appears verbatim, and verbatim support was rarer than a human reader would expect (5 of 19 pages), up to and including the official Devpost page, which states our deadline as "October 30" and therefore does not corroborate the claim as we had written it, "Oct 30". Raw responses and the ablation harness ship in the repo.

What ran live vs recorded — stated plainly. The benchmark numbers above are from a real, recorded live run against Token Factory on 2026-10-28; the harness in the repo re-runs them end-to-end with an API key. The Claim Court gate verdicts were computed deterministically on 2026-10-30 from evidence rows fetched that morning. The reviewer lanes have not yet run over the commerce corpus, and the Court's live rebuttal round has not run; the demo labels both NOT RUN on the cards themselves. The demo page is a live rendering of those frozen, committed run files — every number on it traces to a file in the repo.

What is new in the submission period (significant update). why-agent began as a deterministic reconciler for agent work (Open Agent hackathon, Oct 2026). New since: the Token Factory provider profiles with pinned Nemotron tiers, the tiered reviewer lanes, the Tavily live-evidence lane with its literal-appearance rule, the Super VL lane, the batch-eval harness (cold-load retry, hard budget cap), and the adversarial benchmark with its zero-false-approvals gate. New for this entry: the Claim Court commerce corpus and aspect mapping, the Nemotron Court citation audit, and the Claim Court Case Lab. The kernel is unchanged — the same build is production infrastructure for our agent portfolio, and Claim Court is dual-use: it is the pre-publish check we run on our own ventures' listings and client pages first.

Honest limits.

  • The reviewer lanes are advisory by design and over-approve relative to the reconciler (measured above). Wiring them as gates would be a different, worse system.
  • The commerce corpus is 18 claims, weighted to one real pilot business plus labelled fixtures. It demonstrates the mechanism on commerce claims; it is not a marketplace-scale evaluation, and we don't present it as one.
  • First Token Factory calls after idle can stall for minutes on cold model load; the harness retries, and the receipt says when it did.
  • The checker is literal. It flags restated system vocabulary alongside genuine inventions, and we classify rather than loosen it.

Feedback on the tools — see the tool-feedback field: the same text is pasted there in full.

Built With

  • nebius-token-factory
  • nvidia-nemotron
  • python
  • tavily
Share this project:

Updates

Submission history