Pill Pal

Your medication, simplified.

Comment

Inspiration My grandfather suffered a stroke after struggling to keep up with multiple medications, which made me realize how fragile medication management can be for elderly patients without proper support. That experience inspired me to build Pal a simple system to prevent missed doses and give families visibility and peace of mind. Elderly patients often manage 5+ medications daily with no clear system, leading to missed doses, dangerous interactions, and anxious family members with no visibility into what’s actually happening. We built Pal to fix this gap by creating something that feels as simple as asking a question, but powerful enough to manage an entire medication routine an intuitive companion that replaces confusion with clarity.

What it does Pal lets users scan a pill bottle with their camera and instantly generates a clean, plain-English medication card powered by Claude AI and the FDA API. It automatically checks for dangerous drug interactions, sends browser push notifications as reminders for doses, and gives family members a real-time dashboard showing whether medication was taken or missed. On top of that, a built-in voice assistant allows users to ask questions like “Which medicine was I supposed to take twice today?” or “Is it dangerous to take these two pills on the same day?” completely hands-free, with spoken responses.

How we built it Our system combines a simple user experience with a fairly complex backend pipeline.

Frontend: React + Vite + Tailwind CSS — designed mobile-first with large text, high contrast, and a single primary action per screen to reduce cognitive load. Backend: FastAPI (Python) with async endpoints handling medication processing, scheduling, and dashboard updates.

AI Layer: Claude Sonnet acts as the core intelligence for OCR interpretation of pill bottles, medication explanations, drug interaction detection, and the conversational voice assistant brain, enriched with FDA API data for accuracy.

Voice: Deepgram handles both speech-to-text and text-to-speech, enabling full conversational interaction with medications.

Database: Firebase Firestore powers real-time syncing for the family dashboard and stores medication schedules and adherence history. Notifications: Web Push API + pywebpush delivers background reminders through a service worker, even when the app isn’t actively open.

Challenges we ran into

Getting reliable push notifications across different browsers was the hardest technical hurdle. We discovered that Microsoft Edge relies on Windows Notification Service (WNS), which doesn’t behave consistently with standard Web Push flows, forcing us to prioritize Chrome for full reliability. Another major challenge was keeping the reminder scheduler perfectly synchronized at minute-level precision without duplicate or missed notifications, which required careful deduplication logic and time-window handling. We also had to fine-tune AI outputs to avoid overly complex medical explanations and keep everything safe and understandable.

Accomplishments that we're proud of

We built an end-to-end voice experience where users can speak naturally about their medications and receive spoken, context-aware answers that take into account their full medication list, age, weight, and conditions. We also achieved reliable push notifications that still fire even when the browser tab is closed, which was critical for real-world medication adherence. The real-time family dashboard became a key feature, giving caregivers visibility without needing constant check-ins. Real time data collecting and tailored tips, really make our app stand out compared to others like it.

What we learned

Browser notification systems are far more fragmented than expected, and handling cross-browser behavior requires designing around limitations rather than assuming uniform support. We also learned how important proper async architecture is in Python when multiple services AI, notifications, and database listeners are all interacting at once. On the AI side, Claude’s vision capabilities proved surprisingly strong for interpreting pill bottles, even when images were blurry or poorly lit, which opened up a lot of possibilities for accessibility-focused tools.

What's next for Pal Next, we plan to expand and package it into a real mobile app instead of a responsive web app. We also want to expand Pal into a more proactive health assistant. That includes pharmacy integrations for automatic refill reminders, SMS escalation when doses are missed and not acknowledged by family members, and a full caregiver mode where family can remotely manage medication schedules. We also want to add predictive insights, like warning users when adherence patterns suggest increasing risk or confusion.

Built With

Share this project:

Updates