Uploading Financial Statements
Agents Discovering Information
Ready to Call, All Information Finalized
UI Mockups W/ Notability
System Architecture
Running Back Notability!

Remy

Your bill. Your rights. Your money.

Inspiration

A few months ago, one of our teammates fractured his wrist after falling off an electric scooter. He went to the emergency room, received a couple of X-rays and a splint, and went home.

Then came a bill for $7,420.

As a college student, his first reaction was simple: “I don’t have seven thousand dollars—but I guess I have to pay it.” Only later did we learn that the hospital published a financial-assistance policy that could potentially have reduced the eligible hospital balance to zero.

The help existed. The patient just had to know where to find it, understand pages of eligibility rules, gather the right evidence, call the right department, and say exactly the right thing.

This problem extends far beyond one bill. A KFF survey found that roughly four in ten American adults have some form of health care debt. Dollar For estimates that hospitals annually bill patients for at least $14 billion in care that could qualify for financial assistance. (KFF (https://www.kff.org/health-costs/kff-health-care-debt-survey/), KFF Health News (https://kffhealthnews.org/podcast/an-arm-and-a-leg-podcast-charity-care-nonprofit-dollar-for-medical-bills/))

Hospitals already publish charity-care programs, discounts, collections protections, and payment-plan rules. But those policies are scattered across hospital websites, buried inside PDFs, and written in language most patients never have time to decode.

So we built Remy: an AI agent that helps patients find, verify, and actually use the assistance they may already be entitled to.

From sketches to Remy

Before writing code, we explored several possible agent workflows. Our earliest Notability sessions compared enterprise financial agents, multi- company data workflows, voice-first interfaces, and automated research systems.

Early UI concepts for several possible financial-agent projects

We recorded voice memos alongside the sketches so we could revisit our discussions and evaluate each idea against the same questions: Was the problem meaningful? Did an agent provide real value? Could we access trustworthy data? Would voice improve the experience rather than exist as a gimmick?

Those conversations led us away from a broad financial assistant and toward a much more concrete problem: helping an individual understand and challenge an overwhelming hospital bill.

We then mapped the complete patient journey—from uploading a bill and insurance statement, through parallel financial and policy research, to the approval gate, billing call, outcome review, and follow-up.

Remy architecture, patient flow, and demo-story planning

That flow became the architecture we ultimately built: Claude reads the documents, Nessie helps establish the patient's financial picture, Bright Data finds the hospital's published policies, Cartesia powers the voice interaction, and the patient approves every consequential action.

We also planned the demonstration alongside the product. Our notes connect the personal opening, supporting research, live workflow, and final result so the demo explains not only what Remy does, but why it needs to exist.

Explore our complete Notability design process, including our original discussion recordings Mapping thoughts with notability!

What it does

Drop in your hospital bill and insurance statement. Remy works through five steps, and every step stays open with its full record.

Reads your bill. Claude reads the bill and the insurance statement (EOB) from PDFs or photos.
Review: two agents check what you actually owe, side by side.
- The Nessie agent reconciles the bill against your EOB and your bank transactions (Capital One Nessie). It finds insurance adjustments the hospital never applied, payments it never credited, and duplicate charges.
- The Bright Data agent checks you against the hospital's own financial-assistance policy, which we crawled with Bright Data and verified quote by quote, plus the hospital's real prices. It works out your income from payroll deposits and finds the discount tier and payment-plan terms you qualify for.
- Every check and finding appears live with its proof and source. Tap Ask on any finding to have the agent that found it explain it, or This is wrong to correct it.
The plan. A Decide agent turns the findings into a call plan: the money, who to call, and every ask the evidence backs, biggest money first. You can talk to the planner ("refund first", "I can only do $150 a month") and it makes a new version. It changes strategy, never facts.
The call. When you tap Call now, Remy phones billing. It says it's an automated assistant calling with your authorization, cites the evidence, asks for each correction, the discount and a hold on collections, and gets a case number. It never agrees to pay anything or gives card details.
What's next. After the call, Remy marks each ask: got it, refused, promised later, or no clear answer. It only counts a win it can back with the rep's own words. Then it plans the follow-up call with only what's left, and tracks every promise ("a corrected statement within 5 business days") so it knows when to call back.

How we built it

Money is computed, never generated. Reconciliation, income from payroll deposits, percent of the 2026 federal poverty line, discount tiers, refunds and payment plans are deterministic Python. Claude extracts policy rules only with verbatim quotes, and our code rejects any quote it can't find on the source page. The AI can argue, but it can't invent a number.

Our own agent harness: every tool has a permission tier (read, act, consequential). Consequential tools, like placing a call or saving a payment plan, wait for the patient's approval. Every tool call and result is streamed to the case page and written to an audit log.

Anthropic Claude (Sonnet 5.5 and Haiku 5.5): reads bills and EOBs; extracts policy rules with quotes; runs the live call with streaming tool use; answers "Ask" questions from one agent's evidence only; turns planner chat into strategy changes that our rules check; and reads the finished call to mark each ask, but a mark only counts if its quote really appears in the rep's words.
Bright Data (SERP API and Web Unlocker): finds and fetches each hospital's financial-assistance pages and policy PDFs into a versioned policy database for 12 Bay Area hospitals. Every rule is verified against its source, and the database detects when a policy changes.
Capital One Nessie: the patient's accounts, payroll deposits (income), purchases (payments to the hospital and duplicate charges), and an approved payment plan written back as a bill.
Cartesia: our own phone pipeline on Twilio Media Streams. Ink does streaming speech-to-text with key terms (names, insurer, "EOB"). Sonic streams text-to-speech with a voice and accent picker. A pronunciation layer makes claim numbers, billing codes, drug names, money and dates sound the way a billing rep expects. Claude's reply streams into Sonic sentence by sentence, and we start Claude early on Ink's eager_end. The agent stops talking when the rep interrupts.
Also: real prices for 11 hospitals from their public CMS price-transparency files; FastAPI and WebSockets; a vanilla-JS case page that streams every step live.

Challenges we ran into

Trusting an LLM with money. Our first version let the model state numbers. We moved every figure into code with a source, and made Claude quote policy text word for word.
Latency on a live call. Replies took 1.4 to 2.7 seconds per turn. Streaming Claude into Sonic sentence by sentence, starting early on eager_end, and prompt caching brought first audio under one second.
Hospital websites. Bright Data returns 200 even when the target fails (the real status is in the headers), search sometimes returns only redirect links, and policies hide in PDFs. We crawl each hospital's own site and PDFs and verify every quote.
Knowing what a call actually won. Billing reps say "I'll make a note of that" a lot. The scorecard only counts a win backed by the call record or the rep's own words.
We ran out of Cartesia credits mid-hackathon while measuring latency, so the post-call steps were built and tested against recorded and fixture calls.

Accomplishments that we're proud of

Every number on screen traces to a source: a bill line, an EOB line, a bank transaction, or a policy clause.
12 Bay Area hospitals in a policy database built from live crawls, with every rule verified against its source, and real price tables for 11 of them.
Voice replies in 0.44 to 0.8 seconds after the rep stops talking (phone simulator). In desk mode the median went from 3.0 s to 0.93 s.
281 automated tests, all offline: money math, both review agents, the plan rules, the call scorecard and the follow-up flow.
A full loop: audit → plan → call → score → next call, with a human approving every consequential step.

What we learned

The hospital's own policy is the strongest argument. A billing rep can't argue with their own hospital's written rules, quoted word for word. The hard part isn't the AI. It's getting the right facts in front of the right person, and then holding them to what they promised.

What's real and what's simulated

Real: Bright Data crawls of real hospital policies, real prices from hospitals' public price files, Nessie API records, Claude's reasoning, Cartesia speech-to-text and text-to-speech, the approval gate, the audit log, and the 2026 poverty guidelines.
Simulated: the patients, bills, EOBs and bank histories are synthetic (the PDFs are watermarked SAMPLE). In the demo, a teammate plays the hospital billing rep.

What's next for Remy

A pilot with a patient-advocacy nonprofit, free for patients who qualify for charity care.
Signed HIPAA authorization, a letter fallback and phone-menu navigation, so it works on real hospital lines.
Beyond hospitals: the engine is general. It reconciles the bill against your own records, finds the provider's own rule, calls with the evidence, and holds them to the promise. Next are insurance claim denials (fewer than 1% of denied marketplace claims are ever appealed, KFF), then telecom and utility bills, then business invoices.