Inspiration
Malaria remains one of humanity’s deadliest infectious diseases, especially in rural and resource‑limited communities. Traditional surveillance and triage methods—clinic‑based testing, entomological surveys, and paper‑based reporting—often lag behind real‑time transmission dynamics. I was inspired by the proliferation of smartphones and voice interfaces to create a simple, field‑ready triage tool that leverages voice AI, environmental data, and predictive modeling to accelerate early detection and response in under‑resourced settings.
What it does
VoaProtect transforms spoken symptom reports into actionable risk assessments:
- Voice capture of user symptoms using Vosk speech‑to‑text (multi‑language support).
- Symptom matching against a curated symptom list (e.g., fever, chills, headache).
- Environmental integration by fetching local humidity ($h$) and temperature ($T$) via API.
- Predictive model computes malaria probability: $P( ext{malaria}\mid x,h,T)=\sigma(w_x x + w_h h + w_T T + b),$ where $x$ is symptom count and $\sigma$ is the logistic function.
- Visualization: QR code export for offline scanning and Folium map of risk location.
- Feedback loop: Local health workers can log notes for continuous improvement.
How we built it
- UI: Built with Streamlit; a sidebar for language selection and recording, tabs for Results & Logs, and
st.metricfor clear metric display. - Voice pipeline: gTTS for prompts,
sounddevicefor audio, and Vosk Kaldi models for transcription in English and French. - Modeling: Trained a logistic regression on features $[x,h,T]$, serialized with Joblib; fallback to random risk when offline.
- Mapping: Folium +
streamlit-foliumto overlay markers on interactive maps. - Internationalization: All prompts, labels, and logs dynamically switch between English and French dictionaries.
Challenges we ran into
- Streamlit reruns: Preventing audio prompts from repeating on each widget interaction required careful session‑state management.
- Asynchronous recording: Integrating a 10s non‑blocking audio loop within Streamlit’s synchronous rerun cycle.
- Dependency weight: Bundling Vosk models and geocoding libraries without inflating package size.
- Offline resilience: Designing fallbacks when internet or API access is unavailable.
- UX for low‑literacy users: Crafting clear voice prompts and progress feedback for varied user profiles.
Accomplishments that we're proud of
- Under‑an‑hour prototype: A fully functional demo with voice input, QR export, mapping, and ML in less than one workday.
- Dual‑language support: Seamless English/French switching with no hard‑coded text.
- Offline QR integration: Reports can be scanned and uploaded later, bridging connectivity gaps.
- Environment‑aware risk: First demo I’ve built that fuses real‑time weather data with clinical symptoms.
What we learned
- Streamlit’s power: Rapid UIs with minimal code, but requires session‑state finesse.
- Voice in the field: Vosk can handle noisy, on‑device scenarios with modest hardware.
- Model fusion: Simple logistic regression on multi‑modal data can outperform symptom‑only heuristics.
- Internationalization: Early planning for language dictionaries pays huge dividends.
What's next for VoaProtect: Offline Voice AI for Malaria & Rural Triage
- On‑device inference: Containerize models and run entirely offline on edge devices.
- SMS/USSD integration: Enable non‑smartphone users to access triage via basic phones.
- Geo‑cluster analytics: Aggregate anonymized reports into heatmaps for targeted interventions.
- Expanded language support: Add regional languages such as Malagasy.
- Deep learning models: Explore small CNN/RNN architectures for improved transcription accuracy in noisy environments.
Log in or sign up for Devpost to join the conversation.