clinicFlow: WhatsApp AI Queue for Indian Clinics

AI front desk inside WhatsApp for Indian clinics: patients book, wait at home until a live-traffic 'leave now' ping, send voice notes and get Rx PDFs. Built on AWS EUM Social, SES and AgentCore.

Comment

Inspiration

we spent time at a busy dermatology clinic in Meerut. One doctor runs OPD consults and laser procedures, and the front desk runs on paper slips. The waiting room is full by 10 AM because nobody knows when their turn will come. The receptionist answers "kitna time lagega?" on the phone all day, prescriptions get lost, and patients on a 6-session laser package quietly drop off after session 2 or 3 because nobody reminds them. Lakhs of small clinics in India work this way, and every one of their patients already uses WhatsApp.

So we asked: what if the clinic's front desk lived inside WhatsApp, and an AI agent ran it?

What it does

ClinicFlow is an agentic WhatsApp front desk for small skin & laser clinics. Patients install nothing. Staff learn one button.

For patients, on WhatsApp

  • Tap, don't type: a 10-row WhatsApp list Menu and context-aware buttons (🔄 Refresh wait · 📍 Share location · ⏰ Running late · ✅ I've arrived). Typing works in English, Hindi, Hinglish and Devanagari.
  • Two parallel queues: doctor consults (OPD) and laser/procedures, tokens like O7 and L2, each with a live ETA.
  • 📍 "Time to leave" with Amazon Location Service: the patient shares their location once, ClinicFlow computes live-traffic drive time and re-checks it every time the queue moves. When the remaining wait reaches drive time + 5 minutes they get "🚶 Time to leave" with a map pin and one-tap directions. They wait at home, not in the clinic.
  • 🎙️ Voice-note booking: send a voice note in Hindi or English. Amazon Transcribe auto-detects the language, the agent replies "🎙️ I heard: …" and acts. The recording is deleted right after transcription.
  • 📝 Pre-visit intake: 4 quick questions while waiting, shown to the doctor as an intake card.
  • 🚨 Red-flag triage (English + Hindi): emergencies get call 112 instantly, before consent and before any AI model; urgent symptoms jump the queue and are flagged red.
  • 👨‍👩‍👧 Family booking ("token for my mother"), 🕔 laser time slots from a WhatsApp list with an optional UPI deposit link, ⏰ running late (moves back 2 places instead of losing the token), "It's your turn" alerts.
  • 📷 Photos for the doctor (stored privately in S3, acknowledged with a 👍 reaction).
  • 🧾 Prescription as a PDF: generated in Lambda, uploaded with PostWhatsAppMessageMedia, delivered as a WhatsApp document.
  • 🗂️ My visits & prescriptions: past visits, package progress (▓▓░░░░ 2/6), next follow-up and the last prescription, any time. No more lost prescription slips.
  • 📦 Package & 🔁 follow-up reminders with one-tap Book today; 😊/😐/😞 feedback (happy → Google review link, unhappy → privately to the manager).
  • 🔒 Privacy by design (India DPDP Act 2023): consent before anything is stored, stop reminders, and delete my data (erases profile, visits, prescriptions, photos and chat).
  • 📧 Email confirmations, visit summaries and prescriptions via Amazon SES, with a "Check my queue on WhatsApp" button.

For the doctor and receptionist (dashboard)

  • One-button front desk per queue: ✅ Call next (sends "your turn" and fires leave-now nudges), ⏭️ No-show, + Walk-in.
  • ⏳ Doctor running late: one tap broadcasts the delay on WhatsApp and shifts every ETA.
  • 🧠 Self-learning ETAs from today's real consult times.
  • 👤 Patient profile & ✨ AI pre-read: every visit is saved permanently. Each token row shows "🔁 Visit #5 · last seen 15 Oct · Last Rx: Adapalene…" plus ⚠️ allergy and 📦 package flags; one click opens the visit timeline, photo timeline, packages, follow-ups and a 20-second AI summary for the doctor.
  • 🎙️ Dictated prescription: the doctor speaks, Amazon Transcribe (en-IN) converts it (browser speech as live preview and fallback), the AI formats it and it goes to the patient on WhatsApp, as a PDF and by email. "Review in 2 weeks" schedules the follow-up automatically.
  • 📷 Before/after photo compare, 📺 waiting-room TV screen, live impact tiles (waiting-room minutes saved, reception calls avoided, nudges, no-shows, red flags, prescriptions…) and a 📧 doctor's daily summary via SES at 20:30 IST.

How we built it

Fully serverless on AWS, with AWS Communication Developer Services as the backbone and the agent on Amazon Bedrock AgentCore.

Patient ─WhatsApp─► AWS End User Messaging Social ─► Amazon SNS ─► Lambda gateway
                     (triage → consent → media/voice/location)         │ InvokeAgentRuntime
                                                                       ▼
                                          Amazon Bedrock AgentCore Runtime (session per patient)
                                          ├─ model chain: Amazon Bedrock → Azure OpenAI → router
                                          └─ 13 tools → DynamoDB · SES · Amazon Location
Voice notes / dictation ─► Amazon Transcribe      Location ─► Amazon Location Service (live traffic)
Receptionist ─► Lambda Function URL dashboard ─► SendWhatsAppMessage / PostWhatsAppMessageMedia / SES
EventBridge (daily) ─► reminders Lambda ─► WhatsApp + SES; doctor summary 20:30 IST

AWS CDS usage

  • AWS End User Messaging Social (WhatsApp) is the entire patient interface. 3 APIs: SendWhatsAppMessage, GetWhatsAppMessageMedia (photos and voice notes → S3) and PostWhatsAppMessageMedia (prescription PDFs → WhatsApp). 10 message types: text, reply buttons, list menus, location request, location pin, CTA-URL button, reaction, document, template and read receipts.
  • Amazon SES (v2): confirmations, visit summaries, prescriptions, reminders and the doctor's daily HTML report, with an email → WhatsApp hand-off.

Agentic AI

  • Amazon Bedrock AgentCore Runtime hosts the agent (Python 3.12 direct code deploy); every patient gets an isolated session.
  • The agent has 13 tools that change real clinic state: book_token (OPD/Laser, family), queue_status, running_late, im_here, cancel_token, my_packages, book_package_session, set_travel_time, save_intake, laser_slots, book_laser_slot, my_history, save_email. Its prompt includes the patient's visit history, so it greets returning patients and asks about their follow-up.

Model chain with fail-safes (how Bedrock and Azure are set up)

  1. Amazon Bedrock (Converse API + toolConfig, us.amazon.nova-pro-v1:0, optional cross-account role via STS) is first. A circuit breaker skips it for 10 minutes on throttling / access / quota errors.
  2. Azure OpenAI gpt-6-astra (Responses API with function tools, previous_response_id + function_call_output loop). The key lives in AWS Secrets Manager; a 2-minute circuit breaker.
  3. A deterministic intent router (EN / HI / Hinglish / Devanagari) that calls the same tools.

All three drive the same tools, so clinic state never goes out of sync; every turn logs which engine answered. Safety-critical logic (red-flag triage, consent, privacy) runs in the gateway before any model.

Other services: Amazon Transcribe (voice notes with Hindi/English language ID; doctor dictation), Amazon Location Service (CalculateRoutes, DepartNow live traffic), Amazon DynamoDB (single table: clinic config, per-queue atomic counters, tokens with full lifecycle timestamps, daily stats, patients, permanent visit history, packages, follow-ups, slots), Amazon S3 (private photos, PDFs, temporary audio), AWS Lambda + Function URL (gateway, dashboard, reminders), Amazon EventBridge (10:00 reminders, 20:30 summary), AWS Secrets Manager, IAM, CloudWatch, AWS Budgets. The prescription PDF generator is dependency-free Python.

How to test it

  • WhatsApp +91 72178 11862 and send hi (the sender shows Secureyourhealthdata while Meta's display-name review is pending; it's the same number). Tap I agree, then try: Menu → Doctor consultation → your name → intake → 📍 Share location → a voice note ("mujhe laser ke liye token chahiye") → a photo → Menu → 🗂️ My visits & Rx → Menu → 🗺️ Clinic address → "chest pain, can't breathe" (emergency path).
  • The receptionist/doctor dashboard link and key are in the judges-only submission field. Press ✅ Call next, ⏳ Doctor delayed, open 👤 Patient history, and try 🎙️ Rx dictation on the in-consult row.
  • Full step-by-step guide: TESTING.md in the GitHub repo.

Challenges we ran into

  • New-account Bedrock limits. Our partner account was created during the hackathon and its Bedrock daily token quota is pending AWS account verification. Rather than fake it, we built the model chain above. Honest status: the live agent runs on AgentCore Runtime and reasons with Azure OpenAI gpt-6-astra. Bedrock is first in the chain and takes over automatically once the quota is granted, with no redeploy.
  • Azure's reasoning models reject function tools on Chat Completions, so we moved to the Responses API mid-hackathon.
  • AgentCore Memory failed to provision on the new account ("Failed to provision Vector Store"); the integration is implemented and history falls back to DynamoDB.
  • WhatsApp details: the sender must be the phone-number-id-… form, recipients need E.164 with +, and business-initiated templates wait on Meta business verification (inside the 24-hour window we use interactive buttons).
  • Batch Transcribe inside a messaging flow: we keep voice notes short, auto-detect Hindi vs English, and translate Devanagari keywords so even the fallback router understands a Hindi voice note.

Accomplishments that we're proud of

  • Zero install for patients, one button for staff, and the patient waits at home instead of in a crowded room.
  • An agent that acts: 13 tools that book, cancel, move, check in, take intake, book slots and read history.
  • Deep use of AWS CDS: 3 EUM Social APIs, 10 WhatsApp message types, SES with an email → WhatsApp hand-off.
  • Production thinking: model fail-over, triage before AI, consent and erasure, least-privilege IAM, budget alerts.
  • A real clinic problem, with a customer engagement registered in AWS Partner Central.

What we learned

Messaging-first beats app-first for Indian healthcare: the adoption barrier is the receptionist, not the patient. Design for the AI being unavailable: a deterministic fallback over the same tools made the product dependable, not just impressive. And voice notes matter: many patients would rather speak than type.

What's next for ClinicFlow

  • Turn on Amazon Bedrock (Nova) and AgentCore Memory long-term strategies once account verification completes.
  • Multi-doctor rooms and a clinic-branded WhatsApp sender name; WhatsApp Flows for intake.
  • AgentCore Gateway to expose clinic tools over MCP; payment-gateway webhooks for deposits.
  • A two-week pilot at the Meerut clinic measuring no-shows, phone calls and package completion, then multi-clinic onboarding and an AWS Marketplace listing.

Built With

Share this project:

Updates

Submission history