Pocket Apothecary

Project Logo

Inspiration

Pill-scanning apps answer one question: "What's in this bottle?" But the real danger happens around the bottle. A parent comes home from the hospital with six new prescriptions, a confusing discharge sheet, and a cabinet full of old pills, and nobody checks how it all fits together.

If English isn't the family's first language, the instructions are even harder to follow. We wanted to build something for equal access to safe care: a pharmacist in your pocket that speaks your family's language.

What it does

Pocket Apothecary is a Progressive Web App (PWA): it installs on any phone straight from the browser, with no app store.

Scan: snap a bottle or box, and Gemini vision returns structured data (drug, strength, dose, frequency) with a confidence score.
Confirm: you check and edit what it read. Low-confidence rows are highlighted.
Normalize: RxNorm maps brands, generics, and misspellings to one ingredient (Coumadin, cumadin and warfarin all become warfarin).
Check: a rule engine flags drug-drug, allergy, condition, and duplicate-therapy risks in three severity tiers, and every flag cites its source.
Explain: Gemini rewrites each flag in 60+ languages at a simple, standard, or clinical reading level. ElevenLabs reads it aloud.
Schedule and Passport: a daily schedule that spaces out conflicting drugs, with calendar export. Plus an offline Medication Passport with a QR code for the ER.

How we built it

Frontend: React + TypeScript + Vite, made installable and offline-capable with vite-plugin-pwa.
Backend: Python FastAPI, using the Gemini SDK with a structured JSON response schema.
Drug data: the NIH RxNorm API, with a local brand table as a fallback.
Voice: ElevenLabs multilingual text-to-speech.
Deployment: Docker Compose with nginx in front.

Health data stays on the phone. The Passport QR code holds the data itself, compressed into the link's # fragment, so no server ever sees it.

Our core rule: AI reads and explains; deterministic rules decide what's dangerous. For n medications, the engine checks every pair:

$$\binom{n}{2} = \frac{n(n-1)}{2}$$

That's 15 checks for six prescriptions. Each pair is checked against the rules rather than left to a model's guess.

Challenges we ran into

Trusting AI in a medical app. Letting the model "decide" interactions was tempting and risky. Splitting reading from deciding, and adding a human confirm step, solved it.
Messy names. Brand names, strengths, and OCR typos broke matching until we normalized everything through RxNorm.
PWA gotchas. Service workers only run over HTTPS, so testing on a real phone needed a production build behind an HTTPS tunnel, and we had to keep the service worker from caching our API calls.
Reading-level control made explanations harder to get consistently right.
Schedule logic. Turning "twice daily with food" into real clock times while spacing out conflicting drugs was harder than expected.

What we learned

Structured output is what makes vision models usable in production. A schema beats prompt-begging.
Good healthcare UX means admitting uncertainty: show confidence, ask the user to confirm, cite sources.
Shipping one polished end-to-end story beats twenty half-working features.