Inspiration

Most denials that get appealed are overturned, but few are appealed. In Medicare Advantage in 2024, only 11.5% of denied prior-auth requests were appealed, and 80.7% of those appeals were partly or fully overturned (KFF, Jan 2026). In ACA Marketplace (HealthCare.gov) plans, insurers denied 19% of in-network claims in 2024, and fewer than 1% of those denials were appealed (KFF).

A 2022 federal audit of 250 sampled Medicare Advantage prior-auth denials found that 13% met Medicare coverage rules. One pattern it named: plans requiring an x-ray before approving more advanced imaging, which Medicare rules don't require (HHS OIG, OEI-09-18-00260).

Winning one of these appeals is mostly careful reading. You find the payer's policy as it stood on the date of service, check each requirement against the chart, and quote both exactly. A biller at a three-doctor practice rarely has the hours for that.

The payers already do this work with software. Availity, Cohere Health and Anterior sell criterion-by-criterion evidence review to health plans, and eviCore runs imaging review for them. The provider-side tools we found mostly generate appeal letters and are sold to hospitals and health systems. We wanted the payer's own view of a case, built for the practice.

What it does

Denial Auditor works one kind of denial: outpatient lumbar MRI (CPT 72148). It takes the faxed denial letter and the chart notes and returns a recommendation with evidence.

Reads the fax and triages it. Nemotron 3.5 Super-VL reads the scanned letter. A fixed reason-code (CARC) table sends it one of three ways: an administrative denial gets a fix checklist, a plan exclusion is marked not winnable, and a clinical denial goes on to review.
Fetches the policy in force on the date of service. For Medicare and Medicare Advantage, the CMS Coverage API keeps every version of LCD L34220, so the version is picked by date. A June 2, 2025 service date gets v37, and a July 25, 2026 one gets v40 (effective 10/23/2025). For Aetna, a Tavily search finds Clinical Policy Bulletin 0236 on aetna.com at runtime, and the page is fetched directly.
Splits the policy into rules, in the payer's own words. Code numbers every sentence of the fetched text, and Nemotron 3 Super only picks sentence ids. Each rule's wording is therefore cut from the source, not written by a model.
Checks each rule against the chart. Nemotron 3 Nano pulls chart facts, each with a quote, and code finds every quote in its note. Code, not a model, computes durations from the chart's dates. Two Nemotron 3 Super calls judge each rule independently. If they disagree, Nemotron 3 Ultra decides, and the rule is marked as escalated. A "met" with no verified evidence is downgraded to "missing documentation".
Decides with a fixed rule, not a model. The five outcomes are: appeal, ask the physician, don't appeal yet, fix and resubmit, or not winnable.
Drafts the appeal behind a citation gate. Ultra drafts the letter from pre-verified quotes only. Super critiques it as the payer's medical reviewer would, and Ultra revises it. Export stays locked unless every quoted passage matches the policy fetched for that case, or the chart, word for word. Change "four weeks" to "two weeks" in a quote and that quote is marked void, the sentence the source really contains appears beside it, and export locks.
Asks the physician when the chart falls short. It doesn't write a weaker appeal. It drafts a documentation request that quotes the rule and says: "Please don't add anything that didn't happen; only document care that was actually provided."

Two more modes use the same checks:

Prior-auth check (before you submit). It runs the checks on a request before it goes out. When a waiting period is the only gap, code works out the earliest date the chart can meet it.
Policy watch. A nightly job with no model compares the policy version each open case cites with the live source. It names the rules whose exact wording no longer appears.

What it won't do:

It never submits anything. A person reviews, exports and sends every letter.
Appeal deadlines come from a fixed table checked against eCFR, never from a model: 42 CFR 422.582(b), 42 CFR 405.942(a), 45 CFR 147.136(b) and 29 CFR 2560.503-1(h)(3)(i).
It says what the policy for the date of service requires. It never says a denial was unlawful.

Scope today: lumbar MRI (CPT 72148); Medicare and Medicare Advantage through LCD L34220 for practices in Noridian's 13 states (AK, AZ, CA, HI, ID, MT, ND, NV, OR, SD, UT, WA, WY); and Aetna CPB 0236. Any other payer or state gets a clear refusal, not a guess. That scope is narrow on purpose: it is one policy family we could check end to end against gold answers.

How we built it

Models (all NVIDIA Nemotron, all called through Nebius Token Factory at runtime). We probed each model before choosing it (scripts/probe_models.py, Oct 21).

Step	Model	Why this model
Read the scanned denial letter	Nemotron 3.5 Super-VL	The only vision model on our key. Token Factory lists its image price as $0.
Read a text notice / triage without a reason code	Nemotron 3 Nano, thinking off	In our probe, a small extraction took 0.4 s with thinking off and 4.2 s with it on.
Split the policy into rules (pick sentence ids)	Nemotron 3 Super	7 of 7 gold criteria in 3 of 3 runs. Ultra matched it at about 3× the cost; Nano got 6 of 7, and Lightning was slowest and missed one.
Extract chart facts with quotes	Nemotron 3 Nano	A cheap first pass. Every quote is re-found by code, so an invented one is dropped.
Judge each rule against the chart	Nemotron 3 Super ×2, Ultra on a split	One Nano vote got 16 of 29 recommendations right on our set; two Super votes plus the Ultra referee got 30 of 30. Ultra decided 58 of the 410 rule checks in that run.
Draft, critique, revise the appeal	Ultra, then Super, then Ultra	Ultra writes; Super plays the payer's reviewer.

Nemotron 3.5 Lightning took 50 to 230 seconds even on small prompts in our tests, so it isn't on any request path.

Every model call uses response_format with a JSON schema and is validated with pydantic. Models only ever return text, and code checks every piece of it. Rule wording is cut from the source by sentence id. Chart quotes are located with a substring match, and offsets from the model are never trusted. Durations come from dates, the verdict from a fixed rule, and deadlines from a table.

Policy sources. The CMS Coverage API is the primary source: free, versioned, and returning each LCD version's effective dates. Aetna doesn't publish prior versions of a bulletin, so Aetna CPB 0236 is a dated snapshot. The tool records the fetch date and Aetna's "Last Review" date (04/09/2026 in our showcase run), and labels the result as a snapshot, not a versioned policy. The policy watch re-fetches it and flags changes.

Spend caps in code. A custom httpx transport sits under the OpenAI SDK. It checks the worst-case cost of every Token Factory request against a project cap before sending it, and records the actual cost afterwards. Tavily calls go through the same kind of guard. The caps are $27 for Token Factory and 4,125 credits for Tavily. The development ledger shows about $6.18 spent so far, including every 30-case evaluation.

App. The backend is FastAPI. The frontend is React with shadcn components. Every quote in the UI links to the exact sentence it came from in the policy or the chart. The app runs on Google Cloud Run, scaled to zero, with keys in Secret Manager and the spend ledgers in a Cloud Storage bucket. The policy watch runs as a nightly Cloud Run job with a $0 model cap.

Evaluation. We wrote 30 synthetic cases in 7 groups, each with ground truth (cases/*/scenario.json) and gold policy anchors (eval/gold/). Every letter is read from its scanned image.

	Super ×2 + Ultra referee	Nano, one vote
Triage route	30/30	29/29 ¹
Recommendation	30/30	16/29 (55%)
Gold rule statuses	138/138	119/131 (91%)
Mean cost per case	$0.048	$0.010
Mean time per case	72 s	90 s

¹ One Nano run returned malformed output (a runaway answer cut off mid-JSON) and isn't scored. In the Nano run only fact extraction and matching ran on Nano; model and vote count changed together, so this doesn't isolate either.

These are synthetic cases we wrote, scored against our own ground truth. They show the pipeline does what we designed on the situations we designed. They don't show accuracy on real charts.

Challenges we ran into

Models paraphrase policy text. When we asked for spans, we got near-quotes. Switching to sentence ids made exact wording a property of the design.
"Doesn't apply" isn't "met". Early matchers marked an exclusion that didn't describe the patient as "met". We moved to effect-based statuses with code rules on top, then added the second vote and the Ultra referee. On a five-case pilot the early matchers got 1 of 5 (Nano) and 2 of 5 (Super) recommendations right. Two votes with a referee got 24 of 24 clinical cases, then 30 of 30 on the full set.
Small models are confidently wrong in a specific way. Nano's misses were mostly "blocks" on the red-flag route when the patient simply had no red flags (8 of 18), and "undocumented" for four weeks of care that was in the chart (3 of 18). One extra Super vote cost about 5× more and removed both.
Aetna has more than one route to coverage. Some of our synthetic patients qualified through a route we hadn't planned for. The scenarios were wrong, not the model.
Unicode. Models emit U+2011 non-breaking hyphens. The gate treats hyphen variants as equal, but keeps words, digits, case and dashes strict.
Licences. eviCore, Carelon and ACR terms forbid reuse, so that text is never sent to a model or stored (sources.assert_allowed), and we link to it by name only. UHC was dropped because its imaging criteria are eviCore text.

Accomplishments that we're proud of

An appeal can't be exported unless every quote is found word for word in a source fetched for that case. In the showcase appeal (case 015), all 12 unique quoted passages pass, 22 uses in all.
The policy version comes from the date of service, not from whatever is live today.
It can say "don't appeal yet" or "ask the physician", and it explains why.
Every model call, vote, referee decision and cost is in the case's audit trail.

What we learned

Reasoning models judge one rule at a time well. They're poor sources of truth for quotes, dates and arithmetic. The product got better each time we moved one of those jobs from the model into code.

What's next

Measure what matters with real users. We haven't tested with a biller and haven't measured time saved per denial. The first step is a pilot with a small practice or billing company on de-identified denials, timing a manual appeal against an assisted one, and tracking overturn outcomes.
Before any real patient data: business associate agreements with every vendor, confirmation of Token Factory's data terms, and a commercial AMA CPT licence.
Wider coverage: other Medicare contractors' LCDs, CT and other imaging codes, more commercial payers, and PDF or fax intake.
CMS-0057-F: from January 2027, covered payers must offer a prior-authorization API that lists required documentation. The prior-auth check should read it.