Health Guard AI

Transform prescriptions into smart regimens, scan food for drug interactions, and decode complex insurance policies into clear answers using next-gen AI reasoning

Comment

Inspiration

"We’ve all been there: staring at a doctor’s scribbled prescription, wondering what the side effects are, or standing in a grocery aisle unsure if a packaged food interacts with our new medication."

Healthcare data is fragmented. Patients are overwhelmed. We built Health Guard AI to be the "connective tissue" between your doctor's notes, your daily diet, and your insurance coverage. We wanted to use Gemini 3’s multimodal reasoning to turn complex medical documents into simple, actionable safety alerts.

What it does

Health Guard AI is a comprehensive medical companion with three core pillars:

1. Prescription Intelligence (The "Medical Lens")

  • Instant Digitization: Users upload a photo of a prescription. The app extracts drug names, dosages, and instructions using Gemini Vision.
  • Safety & Education: It automatically fetches FDA Verifications (via FDA API) to check for recalls, displays Health Warnings, and curates YouTube Video Guides for each drug so users know exactly how to use them.
  • Smart Adherence: It converts vague text like "twice a day after dinner" into auto-scheduled calendar alarms.
  • Marketplace: Provides a "Buy Online" feature comparing prices across 4 major pharmacies with one click.

2. The Dietary Defender (Food Scanner)

  • Cross-Referenced Safety: Users scan a food label. The AI doesn't just read ingredients; it cross-references them against the user's active prescription.
  • Real-Time Warnings: It flags "harmful ingredients" or "drug-food interactions" (e.g., "Do not eat grapefruit while taking Statins") and checks for allergens.
  • Nutritional Verdict: Provides a clear Safe / Unsafe compatibility score and an expiry date reminder.
  • Custom Diet Charts: Generates a personalized diet plan based specifically on the user's current medication regimen and lifestyle habits.

3. Insurance Simplifier

  • Policy Decoder: Users upload their entire Health Insurance Policy PDF.
  • 1-Page Overview: Generates a jargon-free summary of what is covered and what isn't.
  • Coverage Chatbot: Users can ask specific questions like "Does this cover my dental implant?" and get instant answers based on the specific policy clauses.

How we built it

We built a multimodal RAG (Retrieval-Augmented Generation) pipeline centered around Gemini 3. We modeled the safety verification as a logic function where the AI evaluates the intersection of patient history and new inputs:

$$ \text{SafetyScore} = f(\text{Ingredients} \cap \text{ActiveMeds}) + \text{AllergyProfile} $$

Tech Stack

  • Frontend: Built with Google AI Studio for a cross-platform, accessible mobile experience.
  • The Brain (Gemini 3 Pro): We utilized Gemini's 1M+ context window to hold the user's entire medical history and insurance policy in active memory.
  • Vision: Used to OCR prescriptions and food labels with high accuracy, even handling cursive handwriting.
  • Reasoning: We used Chain-of-Thought prompting to make the AI "think" before answering: $$\text{Input} \rightarrow \text{Identify Context} \rightarrow \text{Check Contraindications} \rightarrow \text{Output}$$

Integrations

  • FDA API: Connected via function calling to validate drug names and fetch real-time recall status.
  • YouTube: To dynamically fetch credible video guides for medication usage.
  • Backend: A Python/FastAPI server orchestrates the calls between the client, Gemini, and external APIs.

Challenges we ran into

  • The "Handwriting" Hurdle: Doctors' handwriting is notoriously difficult. Standard OCR failed. Switching to Gemini 3 Vision was a breakthrough—it could infer drug names based on context (dosage/frequency) even when the letters were ambiguous.
  • Mapping "Vague" Time to "Specific" Alarms: Converting "take in the morning and night" to concrete system alarms required complex logic. We solved this by having Gemini output a structured JSON schedule object that our app could parse into native notifications.
  • Hallucination Risks: For the Insurance Chatbot, accuracy is critical. We implemented a "Strict Citation" mode where the model must reference the specific page number of the policy PDF when answering coverage questions.

Accomplishments that we're proud of

  • The "Food vs. Meds" Engine: Successfully building a system that warns a user about a specific ingredient in a snack because of a pill they took that morning. That level of contextual awareness feels like magic.
  • The 1-Page Policy Summary: Turning a confusing 40-page legal document into a clean, readable 1-page summary that actually helps users save money.
  • Seamless E-Commerce: Integrating the "Buy Medicine" feature to close the loop from diagnosis to treatment.

What we learned

  • Context is King: The true power of Gemini 3 isn't just answering questions; it's remembering that the user is taking "Amoxicillin" while they are scanning a "Yogurt" label three hours later.
  • Trust through Transparency: Users trust the AI more when we show why a food is unsafe (citing the specific ingredient) or where in the policy a claim is rejected.

What's next for Health Guard AI

  1. Wearable Integration: Syncing with smartwatches to track if the user actually took the medicine (gesture detection).
  2. Family Mode: Allowing a caregiver to manage prescriptions and diet for elderly parents within the same app.
  3. Local-First Scanning: Moving the Food Scanner to Gemini Nano for offline usage in grocery stores with poor signal.

Built With

Share this project:

Updates