MediBuddyAI

An AI-powered app that securely stores medical records and prescriptions, tracks medications, sends reminders, and helps users find nearby doctors and hospitals using location and online reviews.

Comment

Inspiration

Keeping track of medicines, check-ups from prescriptions and searching good doctors and hospitals are the most important and toughest job of life, specially if your loved persons get sick. From this experience, we wanted to build something that will help us dealing these jobs perfectly.

What it does

  • This application takes an image or pdf version of your prescription, extract all the information like medication list, medical checkups and next appointment. Then it sets reminders for taking medications, checkups and appointments.
  • This app also has a feature named AI Care Finder - which takes what you want - a doctor, a hospital, or a diagnostic center. Then it searches online, check reviews and show you top results so that you don't have to search and check reviews manually.

How we built it

  • Mobile: Expo + React Native having scan prescriptions, save medication and checkup data, get reminder notification at selected time, search best doctor, hospital, and diagnostic centers based on selected options, etc. features.
  • Backend: NestJS on Fastify, Prisma + PostgreSQL, in a pnpm/Turborepo monorepo. Zod contracts live in a shared package consumed by both app and API, so the two can't drift apart.
  • Extraction pipeline: a prescription image/PDF is uploaded to a private Azure Blob container, then a background job sends it to an LLM for structured extraction into medicines, schedules and check-ups. The result is stored as a draft - nothing enters as real medicine list until user review and confirm it.
  • Scheduling & adherence: medicines carry timezone-aware weekly schedules; every dose you take, skip or snooze is recorded as an adherence event. Check-ups and appointments have their own lifecycle (scheduled -> booked -> completed).
  • Offline-safe writes: the app sends a client-generated event ID with each action, and the server upserts on it. So a retry on a flaky connection can never double-log a dose.
  • AI Care Finder: Gemini 2.5 Flash with Google Search grounding, plus a grounded OpenAI web search as an env-flagged fallback, sharing one prompt and one verification path.
  • Auth: Firebase, with identity always derived from the verified token. Every query is scoped to the owner.
  • Deployed on Azure: Container Apps, PostgreSQL Flexible Server, Blob Storage.

Challenges we ran into

  • Prescriptions are messy. Handwriting, abbreviations, "1+0+1" dosing notation, missing durations. The fix wasn't a better prompt, it was accepting that extraction is a proposal. Missing fields come back as null and the user confirms everything, so the AI is never the last word on what someone takes.
  • Stopping the AI from making things up. Early Care Finder results were partly fiction where plausible clinic names, invented ratings, reviewers who "might appreciate his gentle approach." We added a grounding gate: a result survives only if it cites a URL the search tool actually returned.
  • "Doctor" means three different things. Searching for a gynecologist returned clinics. Each provider type needed its own prompt guidance.

Accomplishments that we're proud of

  • The full loop works: photograph a prescription -> review the extraction -> confirm -> reminders appear on Today -> mark doses taken. That end-to-end path is the whole product, and it runs.
  • An AI health app that admits what it doesn't know. Null instead of a guessed dosage, empty pros/cons instead of invented ones, zero search results instead of a fake clinic. A confident wrong answer is the one thing a health app can't afford.
  • Nothing is saved without consent. Extraction is a draft, never a decision.
  • Security we can defend: private storage container, ownership-checked 5-minute signed URLs, encryption at rest and in transit, an audit trail on every action, and identity that can't be spoofed.

What we learned

  • Grounding beats prompting. Telling a model "don't hallucinate" doesn't work; refusing to display anything it can't cite does.
  • The AI is the easy part. Timezones, idempotency, and ownership checks took longer than every model call combined - and they're what makes it trustworthy.
  • Constrain the input. Replacing text boxes with pickers improved output quality more than any prompt change.
  • Verify SDK docs; don't trust memory. Two of our slowest bugs were APIs we were sure we remembered right.
  • Cheap guardrails first - a symptom check that returns a safe fallback before any external call is a few lines, and it stops the app ever behaving like a diagnostic tool.

What's next for MediBuddyAI

  • Caregiver mode to manage an elderly parent's medicines and appointments from your own phone. This is the situation that started the project.
  • Adherence insights to refill prediction and "you've missed this dose 4 times this week" from the data we already record.
  • Drug interaction warnings across the medicine list.
  • Verify providers through Google Places, turning attribution into verification.
  • Document deletion and export, so users fully control their medical records.
  • Managed Identity for storage, replacing the account key.
Share this project:

Updates