Inspiration

India already publishes the data you need to check a medicine bill. The National Pharmaceutical Pricing Authority (NPPA) fixes a ceiling price for every scheduled formulation, and the Jan Aushadhi scheme (PMBJP) publishes the MRP of more than two thousand generic medicines. A PIB release from March 2026 says Jan Aushadhi MRPs are generally 50 to 80% lower than comparable branded medicines, and that over 18,000 Kendras are open.

But using that data on a real bill is hard. The ceiling list is a 51-page bilingual gazette notification. Ceiling prices are per tablet and exclude GST, while bills show strips, brand names and GST-inclusive totals. To check one line you have to know that "AUGMENTIN 625 DUO" is amoxicillin 500 mg + clavulanic acid 125 mg, find the right row, convert strip price to tablet price and add the right GST. Most families, and especially older parents who buy medicines every month, never do this.

I wanted a tool that does all of that from a photo, explains every number, and never sends a medical bill to a server.

What it does

  • Reads the bill. You photograph or upload a pharmacy bill (or the medicines page of a hospital bill). Bill Lens straightens the photo, evens out the lighting, removes table borders and reads it with on-device OCR. It extracts every line: medicine name, pack, quantity, MRP, rate, GST % and amount. Every field can be corrected, and you can also type medicines in.
  • Identifies each medicine. A trained matcher maps the printed name, even with OCR errors such as "STAMLO S TAB" for "STAMLO 5", to the brand's salts, strengths and dosage form, using a table of about 244,000 Indian brand entries.
  • Checks the price three ways:
    • against the NPPA ceiling price (S.O. 1575(E) and S.O. 1581(E), effective 1 April 2026). Ceiling prices are per unit and exclude GST, so the app compares the price you paid per tablet with ceiling x (1 + GST), using the GST on the bill or 5% when none is printed;
    • against the MRP printed on the bill;
    • against the Jan Aushadhi generic with the same salt, strength and form, showing "you could save Rs X (Y%) on this line".
  • Summarises what you paid, possible overcharges and possible savings, in large numbers, with a confidence label on each line and the full calculation one tap away (including the S.O. number and row of the notification it used).
  • Says what to do: keep the bill, ask the pharmacist to recheck, complain through NPPA's Pharma Sahi Daam app or portal, call the NPPA helpline 1800 111 255, or find a Jan Aushadhi Kendra.
  • Shares a summary card or WhatsApp message with family.
  • Works in English and Hindi, installs as an app, and works offline after the first visit.

A flag is a prompt to check, not proof, and the app says so.

How we built it

  • Frontend: plain JavaScript with Vite, no framework, as a PWA (Workbox service worker). The heavy work (data loading, index building, image clean-up, matching) runs in a Web Worker so older phones stay responsive.
  • OCR: Tesseract.js (WebAssembly, LSTM English model). The engine and model are self-hosted and cached, so nothing is fetched from a CDN and the app works offline.
  • Image clean-up (my own code): flat-field correction against a blurred, max-filtered background, contrast stretch, skew estimation from projection profiles, and removal of long table rules. In my tests, bordered hospital-style bills went from unreadable to readable after this step.
  • Line parsing: column-header detection, token shapes (batch numbers, expiry dates, pack sizes like 1x15 or 15'S), two-line receipt merging, and an arithmetic search that picks quantity, rate and amount so that qty x rate x (1 - discount) = amount. The same search repairs dropped decimal points ("21254" read for "212.54").
  • Medicine matching: trigram retrieval over about 117,000 brand family words, then a learned scorer over 13 features (OCR-confusion-aware edit distance, strength numbers explained, unexplained extra words like CV or LS, dosage form, pack). The weights were trained in PyTorch with a listwise softmax loss on OCR output from training bills plus simulated OCR noise. When two strengths of a brand fit equally well, a flag is only shown if it holds for both.
  • Data: Python scripts download and parse the NPPA gazette PDFs with pdfplumber, fetch the official Jan Aushadhi "Product & MRP List", and normalise everything to one salt + strength + form key. Brand compositions come from the MIT-licensed Indian Medicine Dataset. Source URLs, dates and file hashes are recorded in the repo.
  • Evaluation: I generated synthetic bill photos (five bill layouts, real brand names, invented prices, fictional pharmacies, perspective, blur, shadows and JPEG noise) with ground truth for every line, and evaluated the full pipeline from the photo with the same engine the app uses. Playwright tests the production build at desktop and mobile sizes, including an offline reload.

Challenges we ran into

  • Turning the gazette into data. The notification is bilingual; I had to pull the English half of each table, split combination medicines like "Amoxicillin (A) + Clavulanic acid (B)" with paired strengths, and convert units (mcg, mg/5 ml) into one key that also matches brand data and Jan Aushadhi names.
  • Per strip or per tablet? Bills mix both. The app decides using the printed MRP, the column header or the ceiling price, and never uses the charged price to guess the pack size, because that could hide an overcharge.
  • Strength confusion. Plain fuzzy matching found the right brand but often the wrong strength (Telma 40 vs Telma 80). On the held-out set, fuzzy matching alone got the salt and strength right for 78.1% of lines; the trained scorer gets 97.3%.
  • Being honest about GST. Ceiling prices exclude GST, and GST on medicines changed in September 2025. The app uses the bill's GST when printed and otherwise 5%, and says which one it used.

Accomplishments that we're proud of

  • The whole pipeline runs in the browser, about 7 MB on first visit, with no server and no API key.
  • Every number on screen can be traced back to a row in an official document.
  • On 80 held-out synthetic bill photos (470 lines, including a layout never seen during development): 93.4% of lines extracted with the correct quantity and amount, 97.3% matched to the right salt, strength and form, and overcharge flags at 88.6% precision and 87.3% recall.

What we learned

  • Simple, checkable arithmetic (qty x rate = amount) is a stronger signal for reading receipts than any single OCR confidence score.
  • A small learned model with the right features beat a much bigger general model for this job, and stays small enough for a cheap phone.
  • Reading the actual notification text matters: "exclusive of GST", "per unit" and "plus GST as applicable" change the answer.

What's next for Bill Lens

  • Test on real bill photos donated with consent, especially faded thermal receipts.
  • Read Hindi and regional-language bills.
  • Add newly notified ceiling prices automatically as NPPA publishes them, and cover liquids and injections more fully.
  • Prefill a Pharma Sahi Daam complaint from a flagged line.
  • Find the nearest Jan Aushadhi Kendra from the results screen.

Built With

Share this project:

Updates

Submission history