ResQ-AI: emergency help on WhatsApp, in Urdu, Hindi and English
Doctor's portal: incoming case offer with a countdown
Serverless and managed on AWS
Amazon Connect calls the emergency contact and speaks the alert in Urdu
Doctor roster: numbers confirmed over WhatsApp before any case is offered
No doctor free: administrators get an alarm and an Amazon Connect call
Case history: every voice note, agent decision and tool call
The doctor runs the case from WhatsApp, and numbers stay private
Registering an emergency contact in your own words
An Urdu voice note, transcribed and triaged to ESI 2 by a Strands agent on Bedrock AgentCore
The first doctor to accept gets the case card and the full context
One tap shares the location with the doctor and the emergency contact
The case is offered to three free doctors at once on WhatsApp

Inspiration

When someone collapses at home in Karachi or Lahore, the person next to them is not going to open a hospital app or fill in a form. They panic, and then they do what they already do with everyone they know: they send a WhatsApp voice note, usually in Urdu.

We wanted the emergency system to meet them there. No app, no sign-up, no English required. One voice note should be enough to get a family member called and a real doctor on the line.

What it does

A patient (or whoever is with them) sends a voice note, a text or a photo of the injury to the ResQ-AI WhatsApp number.

It understands the message. Voice notes are transcribed with Amazon Transcribe, and photos are described by a Bedrock vision model.
It triages. A Strands agent on Amazon Bedrock AgentCore assigns an ESI level from 1 (resuscitation) to 5 (non-urgent), picks the specialty and replies in the patient's own language. AgentCore Memory gives it the patient's history from earlier cases.
It acts on critical cases. For ESI 1 and 2, the agent calls two tools through the AgentCore Gateway at the same time: an Amazon Connect call that speaks the alert to the emergency contact in Urdu or Hindi, and a ranking of free doctors for the right specialty. The patient gets a "Send location" button.
It finds a doctor. The case is offered to three free doctors at once on WhatsApp, with Accept and Decline buttons and a 60 second window. The first to accept gets the case. If nobody is free, administrators get a portal alarm and a phone call until someone takes it.
The doctor works from WhatsApp. Messages, photos, voice notes and the patient's location are relayed through the ResQ number, so neither side shares a personal number. Short commands like @resq video start an Amazon Chime video call the patient joins from the browser. Doctors and admins can also use the web portal.
It never dispatches an ambulance on its own. The shared location goes to the doctor and to the emergency contact, and the doctor calls the ambulance and records it.

First-time users register their emergency contact in their own words ("my brother Ali, 0333 1234567"), typed or spoken. An emergency is never held up by onboarding.

How we built it

WhatsApp runs on AWS End User Messaging Social. Inbound events go through SNS into an SQS queue with a dead-letter queue, and a worker service on ECS Fargate processes them. The API is a separate ECS service behind CloudFront and an Application Load Balancer, and the two talk through SNS so the portal updates live over WebSockets.

The triage agent is built with Strands Agents and runs on Amazon Bedrock AgentCore Runtime. Its tools sit behind the AgentCore Gateway as a Lambda function, and patient history lives in AgentCore Memory. Data is a single DynamoDB table plus an S3 media vault. The portal is React 19 on S3 and CloudFront with Amazon Cognito sign-in. Everything is in Terraform, and GitLab CI deploys through OIDC with no stored AWS keys.

Safety rules live in code rather than in the prompt. The tools take the case ID and ESI level from our code and refuse to run unless the case is ESI 1 or 2. Each tool runs at most once per case, so a retried message never calls a family twice. Patient history can only make a case more urgent. If Bedrock fails, triage falls back to a second model and then to a keyword classifier, and the case is flagged as degraded.

Challenges we ran into

Urdu speech. Amazon Transcribe has no Urdu language code, so we run Urdu voice notes through the Hindi model and relabel them. It works because spoken Urdu and Hindi are so close. Amazon Polly has no Urdu voice either, so the Connect call reads an Urdu message written in Devanagari with the Hindi voice.
One doctor, one case. Offers go to several doctors at once, timeouts fire in the background and doctors accept from two places (WhatsApp and the portal). We used DynamoDB conditional writes on every transition so exactly one accept wins and a stale timeout does nothing.
Never calling twice. SQS delivers at least once, and an emergency system cannot call someone's mother twice because a worker restarted. Idempotency keys per message and per tool fixed that.
WhatsApp's rules. Messages outside the 24-hour window need approved templates, so we built and manage six of them in Terraform. WhatsApp calling needs a higher messaging limit than a new number has, so doctors call the patient from their own WhatsApp through a one-time link for now. Browser recordings from the portal were not delivered as voice notes until we converted them to mono OGG/Opus on the server.

Accomplishments that we're proud of

The whole patient side is plain WhatsApp. A frightened person in Karachi can use it without learning anything new, in the language they actually speak, and the doctor can run the entire case from their own phone.

What we learned

Put the guardrails in code, not in the model. The agent is good at understanding a panicked voice note, but who gets called, how often and when is decided by code we can test (the backend has 697 tests).

A queue-first design was worth it from day one. Webhooks only enqueue, so slow Transcribe jobs or Bedrock calls never drop a message, and failures retry on their own.

What's next for ResQ-AI

Integrating with ambulance dispatch (CAD) partners, so the shared location can go straight to a dispatcher.
Taking WhatsApp calls in the portal once Meta enables calling on our number.
Native Urdu speech recognition as soon as a service supports it.
Packaging ResQ-AI as a repeatable offering for hospitals and rescue services, starting in Pakistan.