Works in 3 different languages - English
Works in 3 different languages - Amharic
It is a referral, not a diagnosis
Works in 3 different languages - Urdu
works offline and saves data on local phone which will be synced when access to connection returns
Has a telegram bot for caregivers to use

Inspiration

Picture a two-year-old with a cough and a fast, shallow breath. The nearest doctor is hours away. The person who sees her first is a community health worker with a paper chart, a phone, and often no signal, or just a worried parent holding a phone.

Pneumonia is still one of the leading killers of children under five. Most of those deaths are treatable. What kills is the delay: waiting at home when you should have gone, going to the wrong place, or a family that can't afford to go and waits too long. In Pakistan the first contact is often a Lady Health Worker. In Ethiopia it's a Health Extension Worker. They do an enormous amount with very little.

We happen to be from these two countries (Biruk from Ethiopia and Rayyan from Pakistan), so this wasn't an abstract problem for us. We kept asking the same question: what if the worker had a calm, reliable second opinion in their pocket, and the clinic knew the child was coming before the child arrived? Aman means safety and peace in both Urdu and Amharic. That's what we were going for.

What it does

Aman is offline screening and referral support, not diagnosis. It connects three people who currently can't talk to each other.

The health worker opens the app (no internet needed), answers the standard child-illness questions, and taps once per breath to count breathing. Aman answers with one of three results: care at home, visit a clinic soon, or go to a hospital now. It explains why in plain words, in English, Urdu or Amharic, and prints a card the family can keep, with the warning signs that mean "come back right away."

The family can send a voice note to our Telegram bot in Urdu, Amharic(included in the demo video) or English. No app to install, no forms. Aman listens, asks a few tap-button questions about anything it missed, and replies with what to do. A second button gives cost advice: which level of care fits, what it's likely to cost, the cheapest safe way to get there, and the emergency numbers. Money never changes how urgent the case is. If the child needs a hospital, Aman says go, and it doesn't send anyone to a cheaper place that can't help.

The clinic sees cases arrive live on a dashboard, most urgent first and youngest child first.

What makes the call:

Only a rulebook decides. The rules are copied from the WHO child-illness guidance and cited in the code. The AI never gets a vote.
If answers are missing, it never says "stay home." It says "see a clinic" and flags the case for a professional to review.
Gemini listens. It writes down what the family said in either of the 3 languages. The same rulebook then decides.
The cough-sound score is shown for information only. It can't change a decision.

How we built it

Monorepo (pnpm + TypeScript) split into shared packages for types, the decision rules, audio analysis, and bundled text in three languages.
Field app: a Vite + React + Tailwind progressive web app with a service worker (Workbox) and on-device storage (IndexedDB via Dexie). It works with the network off, and cases sync when a connection returns. It installs to the home screen like a normal app.
Server: Fastify + zod + SQLite, with live updates over Server-Sent Events. The server re-runs the rules on every case and ignores whatever decision the phone sent.
Voice channel: grammY (Telegram bot) + Google Gemini (@google/genai) to turn speech into structured facts.
Dashboard: Leaflet, with routing that favors a language match..
Cost advice: a module with sourced price estimates for Punjab and Ethiopia, tested so it can never recommend care below what the child needs.
Audio: log-mel features and a tiny model that runs in the browser. A Python pipeline trains it, and a parity test proves the browser and Python compute the same numbers.
Languages: Noto Sans Ethiopic and Noto Naskh Arabic are bundled locally, with right-to-left layout for Urdu, so everything renders offline.
Quality: over 130 automated tests, including a 30+ case clinical table for the rules.

Challenges we ran into

There's no good public dataset for this. The cough recordings we could find are mostly adults, or stethoscope recordings, not children on a phone. Instead of pretending, we trained on generated sounds, labeled the score experimental, and turned off any way for it to affect a child's outcome. A perfect score on made-up sounds means nothing, so we don't quote one. However, fully implementing this is a plan that we have for the near future.
Designing for the worst case. We kept asking "what happens if this part is wrong?" Missing data pushes toward a clinic visit, and the AI stays out of the decision entirely.
Offline everywhere. The app, fonts, model and queued cases all have to work with no signal, and cases must sync without duplicates.
Phones are picky. The microphone needs a secure connection, and the Python and browser audio code must match exactly. That's why we built a parity test.
Words matter in a health app. We spent real time cutting repeated lines and contradictions out of the results and the bot's replies, because a worried parent shouldn't have to read the same warning three times.
Time. Two people, 24 hours, a sleepy night, and a lot of ambition. P.S. thank you for the energy drinks.

Accomplishments that we're proud of

A transparent, cited, tested rulebook that is the only thing allowed to decide a referral.
The whole loop works end to end: an offline field visit, a family voice note in their own language, and a live clinic dashboard that already knows the child is on the way.
Cost advice that helps a family without ever weakening the medical call.
We chose safety over a flashy demo. The riskiest feature is deliberately switched off, and the limits get their own screen.
It speaks three languages, and two of them are written right-to-left or in a script many apps still struggle with.

What we learned

In healthcare, the safest place for an AI is where its worst mistake is "ignored," not "harmful."
Honest limits earn more trust than a big accuracy number.
"Offline" isn't one feature. It touches storage, fonts, audio, models and syncing.
Tests that compare two systems (like our browser-versus-Python check) catch mistakes that make demos look better than they are.
Using AI only to listen keeps a record of what the family actually said, so a person can check it.

What's next for Aman

Real pediatric respiratory audio, collected ethically, before the sound score is allowed any role in a decision.
More languages.
Move the family channel from Telegram to WhatsApp, where families already are, and link the bot's reply straight to the cost screen.
Display live clinic capacity and local fee schedules.