ASHA Voice

Speak a home visit in Telugu, Hindi or English. AI writes the record, rules catch danger signs.

Comment

Inspiration

India has about 1 million ASHA workers. They visit pregnant women, new mothers and babies at home, and fill paper registers late at night after long days. Danger signs like high blood pressure in pregnancy or fever after delivery can be missed when records are rushed. Most ASHAs speak Telugu, Hindi or other local languages, not English, so typing forms on a phone is slow. We wanted the ASHA to just talk, and let AI do the paperwork.

What it does

  • The ASHA taps the mic and describes the visit in Telugu, Hindi or English.
  • Whisper speech AI turns the voice into text, and a language model turns it into a structured record: symptoms, BP, temperature, weight.
  • A separate rule engine, based on Indian MCP card, HBNC and IMNCI guidance, flags danger signs as RED, AMBER or GREEN with plain-language reasons.
  • On RED it offers Call 108, Call PHC and a referral slip.
  • The app plans who to visit next, keeps a register of every person (DOB, phone, address, G P A), and has a supervisor view with a needs-action list, WhatsApp reminders and CSV export.
  • “Ask Didi” answers guideline questions from curated sources.
  • It works offline: if the AI is unreachable, an offline pattern reader still catches danger words, and a visit is never shown as GREEN without the AI having read it.

How we built it

React, TypeScript, Vite and Tailwind as an installable offline PWA, with data stored on the phone in IndexedDB. Vercel serverless functions call Groq: Whisper large v3 for speech to text and GPT-OSS 120B for extraction, family messages and Ask Didi. Names and phone numbers are removed from the text before the language model sees it. The danger-sign rules are deterministic code, kept separate from the AI, and tested with Vitest (38 tests) plus an evaluation set of 30 fictional visits.

Challenges we ran into

Browser voice typing was poor for Telugu and Hindi, so we switched to Whisper. We also found a safety bug in review: when the AI was offline, a dangerous visit could show GREEN. We fixed it with an offline reader and a clear “AI could not read this visit” warning.

Accomplishments that we're proud of

The safety engine never depends on the AI being online. On our small fictional test set, offline rules caught all 18 danger cases with no false alarms. We know this set is small and self-written, not clinical validation.

What we learned

In healthcare, AI should help with understanding language, while life-or-death decisions stay with transparent rules and the human worker.

What's next for ASHA Voice

Testing with real ASHA workers and ANMs, more languages, on-device speech for better privacy, syncing with the supervisor’s phone, and clinical review of the rules.

Built With

Share this project:

Updates

Submission history