Inspiration

Many people take medications prescribed by different doctors. Some also add over-the-counter medicines from the pharmacy shelf. Each doctor may see only part of the list. Over-the-counter medicines are easy to miss. The person taking the pills often has the fullest picture. But without a medical background, it's hard to know what each drug does. It's even harder to know which combinations could be risky.

People could just ask a chatbot. But its answers can change from one try to the next. The evidence behind each claim can also be hard to check. Medication safety is too important for that. We wanted something more dependable. In PillWise, curated data and fixed rules decide the warnings. Every finding links back to its source. We then explain each one in plain language.

What it does

Users photograph their pill bottles or enter medication names, review and correct the list, and then check their medications together.

Recognizes medications: Reads bottle labels and standardizes medication names to identify active ingredients, such as Advil to ibuprofen.
Checks potential risks: Compares medication pairs against its supported interaction data and applies additional rule-based checks for adults aged 65 and older.
Prioritizes findings: Highlights the most important next step and displays medication relationships in an interaction heatmap.
Shows the evidence: Links findings to supporting sources, such as DDInter interaction records or exact sentences from FDA drug labels.
Explains medications clearly: Provides plain-language explanations and displays directions taken directly from the medication label rather than generating dosing instructions.
Prepares users for a pharmacist visit: Organizes questions and provides a plain-language summary and a quick understanding check.
Checks a potential new medicine: Lets users check a medicine against their existing list without changing that list.

If a medicine cannot be matched or checked, PillWise says so clearly. It never treats "not found" as "safe."

How we built it

We divided the work into three roles and developed separate branches in parallel.

Pipeline: A vision model reads medication labels, RxNorm standardizes drug names, and openFDA provides drug-label text for evidence and explanations.
Data and rules: We integrated DDInter 2.0 and implemented rule-based checks using the ONC high-priority drug interaction list and a subset of the 2023 AGS Beers Criteria for older adults.
UI: We built a Streamlit web interface and worked on an Expo app to support a shared experience across web and mobile.

Our core design principle is simple: data and deterministic rules determine the warnings and their severity; AI helps read labels and explain the results. The AI does not invent warnings or independently change their severity. Each finding is designed to include traceable supporting evidence.

Challenges we ran into

Drug names: RxNorm, DDInter, and openFDA do not always use the same naming conventions. A mismatch can prevent an interaction from being detected, so we focused on consistent drug normalization and tested it on our demo medications.
Verifying quotations: AI-generated text can accidentally include words that do not appear in a source. We use a validation step to check whether quoted FDA label text appears in the original label, rather than relying on the model alone.
Look-alike names: A misread label can turn a medicine into the name of another real drug. Database lookup alone cannot catch every such error, which is why users must review and confirm their medication list before running checks.
Plain language: Medication safety information can be difficult to understand. We worked to simplify explanations while preserving their meaning and keeping medication directions faithful to the original label.
Integration: Developing separate components in parallel and merging them required agreeing on shared rules and interfaces up front and double-checking them before we started.

Accomplishments that we're proud of

Built a medication-checking workflow that combines label reading, drug-name normalization, rule-based checks, and evidence retrieval.
Integrated curated medication data and implemented checks for drug interactions and selected older-adult medication risks.
Designed evidence-backed findings that connect warnings to their supporting sources and versions.
Made user confirmation and explicit handling of unchecked medicines part of the workflow.
Built a user interface that brings medication findings, plain-language explanations, pharmacist questions, and source information together.

What we learned

We learned how to actually use AI well. Before letting AI touch anything, we wrote a lot of strict rules for how drug names get normalized and which interactions get flagged. We went back to the original papers and guidelines ourselves and checked each rule by hand. In the end, it worked like a harness. AI does the reading and explaining inside it, and the rules keep it in check. That's also what lets us answer the question: "Why does PillWise say this?"

We also learned that saying "we couldn't check this" is not a failure. In healthcare, an honest statement of uncertainty is more responsible than a confident answer without sufficient evidence.

What's next for PillWise

Improve privacy: Explore on-device label reading and remove identifying details, such as names and prescription numbers, before information is uploaded.
Support local models: Further evaluate local models for medication-label reading to reduce dependence on external AI services.
Expand language support: Make medication information accessible to more people through additional languages.
Add personal context: Explore checks that account for diagnoses, allergies, and other relevant health information.
Bring PillWise to mobile: Build a full native mobile experience and, if the opportunity comes, release it on the App Store and Google Play.