BillShield: Private Medical Bill Audits

Find overcharges on a medical bill without your identity leaving your device: on-device Gemini Nano scrubs PII, deterministic checks audit the bill, the cloud sees only redacted findings.

Comment

Inspiration

Medical bills are often wrong: duplicate lines, lab panels billed again as individual tests, math that doesn't add up, charges dated after discharge. They are also the most identity-dense document most people own: name, date of birth, medical record number, insurance member ID, sometimes an SSN. Pasting one into a cloud chatbot to "check it" means handing all of that over. We wanted a tool a patient or caregiver could actually trust, and hybrid inference made it possible.

What it does

BillShield turns a text itemized statement into disputable findings and a ready-to-send dispute letter. Its privacy model is enforced in code, not just promised:

  1. Rules scan: deterministic detection of labeled names, DOB, MRN, member and account IDs, SSN, phone, email, address, and Luhn-valid card numbers.
  2. On-device scrub: Firebase AI Logic with InferenceMode.ONLY_ON_DEVICE (Gemini Nano in Chrome) proposes extra identifiers hiding in free text. We use ONLY_ON_DEVICE, not PREFER_ON_DEVICE, because raw PII must never fall back to the cloud. Proposals are applied only if the exact string exists in the bill, so the model can't hallucinate edits.
  3. Deterministic audit: seven rules (duplicate, line math, subtotal reconciliation, NCCI-style unbundling, double E/M, post-discharge, price outlier) find the money. The model never decides what is wrong with your bill.
  4. Leak gate: the outbound payload is re-scanned for every pattern and every original identifier. One hit blocks the cloud call.
  5. Hybrid letter: the redacted findings go to InferenceMode.PREFER_ON_DEVICE, so Gemini Nano drafts the letter if the browser has it, and cloud Gemini (model name from Remote Config) does otherwise.
  6. Fact check + local re-insertion: deterministic code verifies that every dollar amount and billing code in the letter traces to a finding, with no invented placeholders. Then the real name and account number are put back locally, from a vault that never leaves the tab.
  7. Privacy receipt: the exact outbound bytes, their SHA-256, which inference source ran each step, and the App Check status.

How we built it

  • Next.js 15 on Vercel, TypeScript, Firebase JS SDK v13.
  • Firebase AI Logic (Gemini Developer API backend): getAI, getGenerativeModel with ONLY_ON_DEVICE and PREFER_ON_DEVICE, initializeDeviceModel() on user activation, response.inferenceSource.
  • App Check: ReCaptchaV3Provider by default, ReCaptchaEnterpriseProvider via env.
  • Remote Config: cloud model name and audit threshold.
  • Firestore: anonymous aggregate stats only, with locked-down rules deployed.
  • A seeded synthetic-bill generator with ground truth (planted errors plus every PII string) drives unit tests and a published eval page.

Measured results (misses included): https://billshield-kappa.vercel.app/eval

  • Rules layer, 40 synthetic bills: 7 rules at 100% precision and recall on planted errors; 0/11 clean bills flagged; 0/440 labeled identifiers leaked. 67/68 free-text identifiers leaked without the on-device model. That gap is why the Nano pass exists, and we publish it rather than hide it.
  • Cloud letter path, 12 bills on the live site: v1: 4/12 letters accepted (the model invented [DATE_1] placeholders; the checker caught it). v2: prompt fixed, and the remaining rejections were a bug in our checker (whole-dollar amounts), now fixed. v3: 12/12 accepted, 12/12 redaction-guard passes.
  • The eval also caught two bugs early: a generator bug and a double-counted E/M rule (10 false positives, now 0).

Challenges we ran into

  • App Check registration: the Firebase console blocked registering the web app's reCAPTCHA v3 secret (permissions message), and Enterprise needs its own key. App Check is fully integrated in code, but registration is still pending, so AI Logic enforcement is off for now. While the change propagated we saw 401 Firebase App Check token is invalid. The app now retries once without the header and otherwise falls back to a deterministic template; the receipt reports exactly what happened.
  • Choosing the hybrid mode per step: PREFER_ON_DEVICE is the right default for most features but the wrong one for raw medical PII. Splitting modes by data sensitivity was the key design decision.
  • Models drift: Gemini 2.5 Flash-Lite gets rerouted from Dec 1, 2026, so the cloud model is gemini-3.5-flash-lite, set via Remote Config so it can change without a redeploy.

Accomplishments that we're proud of

  • An end-to-end workflow where every model output is checked by deterministic code.
  • Honest, reproducible evals with the misses on the page.
  • A privacy receipt that makes "what left my device?" a verifiable question.

What we learned

Hybrid inference is most powerful as a routing policy keyed on data sensitivity, not only as an offline fallback. Small on-device models are a great privacy filter in front of larger cloud models.

What's next for BillShield

  • Model-on eval numbers with Gemini Nano in desktop Chrome.
  • A Chrome extension that scrubs any open bill tab or PDF, and an Android client using the same Firebase AI Logic hybrid SDK.
  • Finish App Check registration and enforce it; Firestore with locked-down rules deployed.
  • EOB-vs-bill reconciliation and a negotiated-rate lookup.

Built With

Share this project:

Updates

Submission history