Inspiration
Growing up in an Indian household, I watched my grandparents manage 4 to 8 daily prescription pills. It was a constant struggle: (Which pill is for blood pressure? Which one needs to be taken before food? Are these two medicines safe together?)
Every pill-reminder app on the market assumes the user is a tech-savvy adult who can read English, navigate complex app menus, and respond to smartphone push notifications. But the highest-risk users — elderly parents, rural families, and non-literate/ less-literate patients — who don't use smartphone apps.
I asked myself: (What if we split the roles?) Let the tech-savvy caregiver (an adult child or nurse) snap a photo of the medicine strips using AI, while the elderly patient simply receives a warm, automated phone call in their mother tongue (Hindi/English) and presses 1 to confirm.
That inspiration became -DawaiSaathi ("Medicine Companion").
What it does
DawaiSaathi simplifies daily medication setup and safety:
-Photo Strip Scanner: Caregivers upload up to 5 medicine strip photos. The system extracts the brand name, active salt composition (e.g. Telmisartan 40mg), price, expiry date, batch number, and manufacturer.
-Multilingual Voice Reminders: Places automated phone calls using Twilio and Gemini TTS (Sulafat voice profile). The patient hears a clear Hindi or English voice prompt and presses 1 on any keypad to confirm the dose.
-openFDA Safety Check: Cross-checks active salts against openFDA label data to warn caregivers about dangerous drug interactions like Warfarin + Aspirin.
-Jan Aushadhi Generic Savings: Pairs branded drugs with identical generic substitutes under India's Jan Aushadhi scheme, showing exact monthly savings.
-Household Roster: Lets co-caregivers manage patient schedules together with owner, caregiver, or viewer roles.
How we built it
-App & PWA: Next.js 15 App Router and Tailwind CSS, running as a PWA and packaged into signed Android APK releases on GitHub.
-AI Vision & Extraction: Google Gemini 2.0 Flash (gemini-flash-latest) and Groq (llama-4-scout Vision & llama-3.3-70b) with dual-verify cross-checking.
-Voice Generation: Google Gemini 2.5 Flash Native TTS for natural Hindi/English audio with automatic retry backoff.
-Infrastructure: Cloudflare Workers via OpenNext, Cloudflare D1 (SQLite), R2 private storage, and Supabase Auth + RLS Postgres database.
-Telephony: Twilio Programmable Voice with TwiML webhooks.
Challenges we ran into
-Voice Generation Quotas: Free inference APIs often served robotic audio or hit rate limits. We switched to streaming 24kHz raw PCM directly from Gemini's native audio model with automatic exponential backoff retries (1.5s, 4s, 8s) for 429 errors.
-API Key Header Formats: Gemini's newer AQ. key format rejected standard Bearer headers. Rewriting our Gemini client to use Google's native REST endpoints with x-goog-api-key headers fixed authentication across Cloudflare edge workers.
-Blister Pack Glare: Foil packaging reflects daylight heavily. We tuned the vision prompts to reliably target printed batch details, expiry dates, and salt names despite reflections.
Accomplishments that we're proud of
-Voice Generation Quotas: Free inference APIs often served robotic audio or hit rate limits. We switched to streaming 24kHz raw PCM directly from Gemini's native audio model with automatic exponential backoff retries (1.5s, 4s, 8s) for 429 errors.
-API Key Header Formats: Gemini's newer AQ. key format rejected standard Bearer headers. Rewriting our Gemini client to use Google's native REST endpoints with x-goog-api-key headers fixed authentication across Cloudflare edge workers.
-Blister Pack Glare: Foil packaging reflects daylight heavily. We tuned the vision prompts to reliably target printed batch details, expiry dates, and salt names despite reflections.
-Passing 176 automated unit tests covering validation, gateway proxying, D1 fallback, and dual-verify logic.
-Building an automated GitHub Actions pipeline that compiles Android APKs on every release tag.
-Making something an elderly grandparent can use without learning how to navigate a smartphone.
What we learned
-Voice calls remain the most accessible interface for elderly and non-literate patients.
-Grounding vision model outputs with openFDA label data prevents dangerous AI hallucinations.
-Building on edge workers (Cloudflare Workers + OpenNext) keeps multi-model AI workflows fast and cheap.
What's next for DawaiSaathi
-WhatsApp Missed-Dose Alerts: Sending text updates to caregivers when a patient misses a call.
-Jan Aushadhi Pharmacy Ordering: Direct re-ordering of low-stock generic medicines.
-Regional Language Expansion: Adding native spoken voice support for Tamil, Telugu, Marathi, and Gujarati.
Built With
- apk
- cloudflare
- gemini
- groq
- next.js
- supabase
- twilio
Log in or sign up for Devpost to join the conversation.