MedExplain

Inspiration

Prescriptions and lab reports are full of jargon — abbreviations, drug names, reference ranges — that most patients can't parse on their own, and there's rarely time with a doctor to get it explained. We wanted to close that gap instantly, for free, using AI.

What it does

MedExplain takes a photo of a prescription or lab report and turns it into something a patient can actually understand: a plain-language summary, definitions for medical terms, values worth double-checking with a doctor, and specific questions to ask at the next visit.

How I built it

The frontend runs OCR client-side using Tesseract.js to extract text from the uploaded image. That text is sent to a small Node/Express backend, which passes it to the Google Gemini API with a structured prompt asking for a JSON response: summary, key terms, flagged values, and suggested questions. The result is rendered back to the user in a color-coded, chart-inspired UI.

Challenges

OCR output from real-world photos is messy — inconsistent spacing, misread characters, garbled drug names. We had to prompt the model to interpret likely OCR errors (e.g. a misspelled medication name) rather than fail on them, which turned out to be one of the more useful things it does.

Accomplishments that I am proud of

Getting the model to correctly reconstruct a misspelled medication name from noisy OCR text, and to catch real inconsistencies in a report (like an incomplete blood pressure reading) rather than just summarizing blindly.

What I have learnt

How much OCR noise affects downstream LLM output, and how important structured prompting (explicit JSON schema, explicit "don't diagnose" rules) is for making an AI tool in a health context both useful and responsible.

What's next for MedExplain

Multi-language output for non-English speakers, and trend tracking across multiple reports over time (e.g. flagging that a value has been rising across visits).

AI Tools Disclosure

• Google Gemini API (gemini-3.6-flash) — the core AI/ML component of the project. Given OCR-extracted text from a medical report, it generates a plain-language summary, defines medical terms found in the report, flags values or entries worth double-checking with a doctor, and suggests specific questions to ask at the next visit. It's prompted with a strict JSON schema and explicit instructions never to diagnose.
• Tesseract.js — used for client-side OCR to extract raw text from the uploaded report image before it's sent to Gemini.
• Claude (Anthropic) — used during development to help scaffold the app, debug issues (including an OCR-to-JSON truncation bug and a deprecated model name), and refine the UI.

Built With

• css
• express.js
• google-gemini-api
• html
• javascript
• node.js
• ocr
• tesseract