PillPal

💡 Inspiration

Prescription labels are small, dense, and easy to misread. The information that matters most, like serious side effects, what to avoid, and dangerous drug interactions, is buried in multi-page FDA inserts that almost nobody reads. For older adults and caregivers managing several medications at once, that gap can be dangerous. We wanted to build something that makes a pill bottle as easy to understand as a text message.

💊 What it does

PillPall turns a photo of a prescription bottle into:

• Clear instructions on how and when to take the medication
• A daily dose schedule with reminders
• Side effects, split into common ones and ones that need help right away
• What to avoid, such as alcohol, certain foods, or activities like driving
• Interaction warnings across every medication you scan

Every detail the app reads from the label is shown to the user to confirm before anything else happens.

🛠️ How we built it

• Frontend: Next.js + Tailwind, designed mobile-first so anyone can open it from a link or QR code
• Label reading: A vision-capable AI model extracts the drug name, strength, directions, and refills from the photo as structured JSON
• Name normalization: The NIH's RxNorm API converts brand names, abbreviations, and typos into a standard generic name
• Safety data: The openFDA Drug Label API provides official label sections for adverse reactions, warnings, contraindications, and interactions
• Plain-language layer: The AI rewrites the FDA text in simple language, using only the text we pass in, and says "Not listed. Ask your pharmacist." when a section is missing
• Hosting: Vercel

For interaction checking, every pair of scanned medications has to be compared. With $n$ medications, that's

$$\binom{n}{2} = \frac{n(n-1)}{2}$$

pairs, so someone on 5 medications has $10$ possible interactions to check, and someone on 8 has $28$. That's exactly the kind of problem a person can't realistically keep track of, and a computer can.

🧗 Challenges we ran into

• Real-world photos are messy. Labels are curved, glossy, and often partly covered. We added a confirm screen and a retake prompt whenever a field came back unclear instead of guessing.
• Drug names are inconsistent. A bottle might say a brand name, a generic name, or an abbreviation. Routing everything through RxNorm before querying openFDA fixed most failed lookups.
• FDA labels vary a lot. Some drugs have detailed interaction sections; others are missing sections or return multiple labels. We had to handle gaps gracefully.
• Keeping the AI honest. Our biggest design decision was never letting the model generate medical facts from its own knowledge. It only simplifies official text we provide.

📚 What we learned

• How to chain multiple APIs (vision AI → RxNorm → openFDA → AI summarization) into one reliable pipeline
• That in health tech, trust and safety are features, and designing for "what if the app is wrong?" shaped almost every screen
• How to prompt an AI model for strict, structured JSON output and validate it before showing anything to a user

🚀 What's next for PillPall

• Pharmacist review mode to verify a patient's full medication list
• Read-aloud mode and translations for users with low vision or limited English
• Caregiver sharing so family members can see a loved one's schedule and warnings
• Refill reminders based on the fill date and quantity on the label

Built With

• node.js
• python