Inspiration

A US medical bill and the insurer's Explanation of Benefits (EOB) arrive separately. They show different totals for the same visit, and they describe the care in billing codes. To work out what you owe, you have to line the two documents up, decode each charge and then call a billing office with the right question. That is hard for anyone. It is harder if English is your second language, if you have low vision or low literacy, or if you are handling a parent's or a child's paperwork.

We wanted a tool that does the tedious part with you. It reads both documents, shows where every number came from, points out what is worth asking about, and lets you talk it through out loud in your language. It should never pretend to know more than the paper says.

What it does

Footnote turns a bill and its EOB into a case you can understand, check and act on. It runs on iOS, Android and the web from one Expo codebase.

Import. Photograph pages with the in-app camera (up to 10 pages), pick photos, or choose PDFs. Footnote keeps the originals on the device. You can open them from the verification screen and from each finding.

Read. Gemini reads all pages in one request. It labels each file as a bill, an EOB or a plan document. It extracts the provider, dates, line items with billing codes, and the statement and EOB amounts separately. Anything not printed stays blank; a missing amount is never shown as zero.

Verify. You compare each value with the original, correct it and confirm. Nothing is compared until you confirm. If your confirmed amounts differ from the reading, Footnote pauses the original reading and the live call's copy of the bill, so neither describes old numbers.

Review. Footnote shows the statement balance, the EOB's patient responsibility and recorded payments separately. It lists questions worth asking, including lines Gemini flagged, such as repeated charges or vague fees. It compares each confirmed billing code with Medicare's national rate from the CMS 2026 Physician Fee Schedule. A difference is a question to ask, never proof of an error. When the reference covers only the doctor's part of a hospital service, Footnote asks a question and shows no ratio.

Listen. An ElevenLabs voice reads the explanation and the price findings in the chosen language. The app requests the reading while you verify, so Listen starts at once.

Talk it through. Tap the orb to talk with a private ElevenLabs agent that runs Gemini and knows the saved bill. It answers in 29 of the app's 31 language choices (not yet Hebrew or Thai). It shows live captions, highlights the line it is discussing and accepts typed questions when the room is loud. Its instructions say a price difference is never proof that a charge is wrong, and that it must never tell you not to pay.

Act. Prepare an editable billing-correction request or insurer appeal. Copy it, export a PDF or share it, and send it yourself. Footnote records a submission only when you mark it, and keeps follow-up reminders and notice dates on the case timeline.

How we built it

Footnote app: Expo SDK 57, React Native, TypeScript (iOS, Android, web) | pages, confirmed codes, saved-analysis IDs live audio + captions (WebRTC) v | FastAPI backend (Python) ----> Gemini API: read, explain, translate | |----> Snowflake: Medicare national rate per billing code | |----> Tiger Data (PostgreSQL): saved analyses, claim timeline | '----> ElevenLabs: readings (TTS), questions (Scribe STT), | short-lived agent tokens ---------------> ElevenLabs agent (Gemini 3.5 Flash) <-'

Challenges we ran into

The main challege we ran into was the integration of the frontend and backend. The backend features were developed separately fron the front end so it took quite a while to fully integrate them together. On top of that, we had to find and fix bugs that showed up after integration to ensure it worked properly.

Accomplishments that we're proud of

Footnote was our team’s first hackathon, and we’re proud to have built a working demo of an idea that matters to us. We brought the app’s pieces together, learned to collaborate under a deadline, and made something people can try. We feel like this app actually has a clear purpose that is useful to people in need of it.

What we learned

We learned how to connect services through APIs and make them work together in an app. We gained experience building with Expo, connecting the app to a backend and database, using Tiger Data to store information, and integrating ElevenLabs to add voice features.

What's next for Footnote

Match bill and EOB lines automatically (by date, code, units and amount), so repeated charges and balance gaps are calculated, not only flagged by the model. Measure accuracy on a labeled set of synthetic documents, including blurry photos, repeated lines and document-injection attempts, before claiming reliability. Protect cloud data: per-case authorization without forcing accounts, deletion across providers, and limits on cached audio. More voice help, as candidates we have not built: read-along highlighting and a practice call with a role-play agent.