Inspiration
Medical bills are hard to read, and LLMs are confidently wrong about money. A model that explains a $551 ER bill as "$551 with a $50 refund" sounds helpful and is dangerous. In a KFF survey, 43% of U.S. adults said they had received a medical or dental bill they thought contained an error (https://www.kff.org/private-insurance/could-consumer-assistance-be-helpful-to-people-facing-medical-debt/). The fix isn't a bigger model. Arithmetic should never be the model's job, and the model's words should be checked before a patient sees them.
Who it's for
Patients reading a claim or an explanation of benefits, and the billing-office and patient-navigation staff who answer their calls. The output is a short explanation of what was billed, what the plan allowed, and what the patient owes and why.
What it does
- Deterministic ledger. The claim's lines and plan (deductible, coinsurance, copay, out-of-pocket max) go through a billing engine (
billpilot/billing.py) that produces per-line billed / allowed / patient-owes amounts and the total. - Writer: Nemotron 3 Nano on Nebius Token Factory gets only the ledger text and writes a plain-English explanation at temperature 0.
- Auditor: Nemotron 3 Super on Nebius Token Factory gets the ledger and the draft and returns strict JSON (
ok,problems), flagging any amount, claim or tone that doesn't match the ledger. - Guardrails. Regex checks run last and reject any dollar amount not present in the ledger, plus any draft that doesn't state the total.
- Decision. The Nemotron draft is shown only if the auditor says ok and the guardrails pass. Otherwise the patient gets a safe template built from the ledger. It never "retries until it passes": a failed audit is a signal, not noise. For an EOB document, Nemotron 3 Nano also extracts each field with a verbatim quote, and code checks every quote against the document before recomputing the amount owed.
How we built it
- Two Nemotron models, two jobs.
nvidia/nemotron-3-nano-30b-a3bis the cheap, fast writer.nvidia/nemotron-3-super-120b-a12bis the auditor, with the reasoning to catch an inconsistency. That's the "Nano/Super for fast calls… make your credits stretch" pattern Nebius recommends. - Token Factory's OpenAI-compatible API. Calls go to
https://api.tokenfactory.nebius.com/v1(nemotron.py). Model IDs are environment variables, so swapping models is a config change. - Measured cost and latency per explanation on Token Factory (10 synthetic claims, 2026-10-24; means, output includes reasoning tokens):
| Model | Role | Tokens in/out | Latency | Cost |
|---|---|---|---|---|
| Nemotron 3 Nano | writer | 161 / 944 | 7.9 s | $0.00024 |
| Nemotron 3 Super | auditor | 254 / 204 | 1.9 s | $0.00026 |
| Total per explanation | 9.8 s | $0.00050 |
- Demo: https://igoreduardoai.github.io/claimclarity/ (replays the recorded run; no live calls). Code (MIT): https://github.com/igoreduardoai/claimclarity
Design: one claim, one honest answer
The demo page (a replay of the recorded run) is for a patient or a billing-office agent:
- Pick a document. The EOB text with every evidence span highlighted; each extracted field links to its line.
- Conclusion card. Match, mismatch (stated vs recomputed amount, with the line) or the exact missing fields; no amount is guessed.
- Explanation and run cards. The patient text with badges (Nemotron or safe template, auditor verdict, guardrails), plus model IDs, tokens, latency and cost per call.
A billing-office agent can paste the explanation into a patient message with confidence. If the badge says template, the reason tells them what the model got wrong.
Trust by design
- The model never computes money. Every amount comes from the deterministic ledger.
- Two independent checks. One is a model-based audit (Nemotron Super) and one is rule-based (guardrails). Both must pass.
- Fail closed. Malformed auditor JSON, a missing API key, an audit failure or an unsupported amount all lead to the safe template.
- Transparent output. Each response reports
source(nemotron or safe-template), the auditor's verdict and any guardrail violations, so staff can see why.
Accomplishments that we're proud of
Every conclusion shows where it came from. In a live run on Nebius Token Factory, a synthetic EOB goes in, Nemotron 3 Nano extracts each field with a verbatim quote, and code checks every quote against the document (line and character span), recomputes what the patient owes, and compares it with the amount the EOB states. 9/9 runs (3 documents × 3) reached the right outcome: a match at $551.00, a mismatch caught (the EOB says $601.00 on line 10; its own terms give $551.00), and an incomplete document that names exactly what's missing (coinsurance, out-of-pocket maximum) instead of guessing. Median 16.0 s and $0.00087 per document. It explains and flags; a person decides what to do.
8/8 offline tests with mocked Nemotron responses (5 pipeline, 3 document journey):
- a correct draft is shown;
- a draft that invents a $50 refund is caught by the guardrails (
unsupported_amount:50.00) and replaced; - an auditor "not ok" leads to the template;
- malformed auditor output leads to the template;
- no model available leads to the template, with the correct $551 total.
Live run on Token Factory over 10 synthetic claims: 10 shown as Nemotron drafts, 0 fallbacks. Auditor stress test (30 calls: a correct text plus an inflated total and an invented $50.00 refund per claim): Nemotron Super flagged 20/20 injected errors and raised 1 false alarm in 10 correct texts (it objected to a line item omitted from the summary). The guardrails missed 1 of 20: a $50.00 "refund" on a claim where $50.00 is a real ledger amount. Super caught it, which is why both checks are required. Total spend for all live runs today: about $0.03.
What we learned
Arithmetic should never be the model's job, and the model's words should be checked before a patient sees them.
Challenges
- Getting the auditor to return strict JSON reliably. Anything else is treated as a failure.
- Deciding that a failed audit must fall back rather than retry.
- Keeping the writer from adding "helpful" amounts such as refunds, estimates or discounts that aren't in the ledger.
Limitations
- Synthetic claims and text EOBs only; no PDF parsing yet.
- English output only.
- Not medical, legal or billing advice.
- The guardrails check amounts and the presence of the total, not every phrase.
- The demo page replays the recorded Token Factory run (no live calls), so the API key is never exposed.
What's next
- EOB PDF ingestion.
- Spanish and Portuguese output (the auditor checks the translation against the same ledger).
- Testing with billing-office staff.
- A batch mode on Token Factory for a whole day's statements.
Run it
python3 test_pipeline.py && python3 test_journey.py # 8/8 offline, no key
NEBIUS_API_KEY=... python3 journey.py docs/eob_er_mismatch.txt
Team
Solo build by Igor Eduardo (Austin, TX): design, code, evaluation and video. Founder of Cortexa Social, Inc.; Antler Founder Residency (F26, Austin). AI coding assistants were used during development.
Video note: the narration is a synthetic text-to-speech voice (en-US-AndrewNeural); visuals are screenshots of the recorded Token Factory run from 24 Oct 2026. All claims and EOBs are synthetic.
Built With
- nebius-token-factory
- nemotron-3-nano
- nemotron-3-super
- nvidia-nemotron
- python
