-
-
A photo showing swelling in the patient's legs, used to document and track physical symptoms in the app.
-
Pharmacy_Clue_BP_150_95 Description: A digital blood pressure reading, logged during a morning pharmacy check-in.
-
Clinical Records: Instant digitization of vitals and handwritten health cards via multimodal input.
Mpingo: Protecting Life through Multimodal AI Triage :contentReference[oaicite:0]{index=0}
Inspiration
Mpingo means "life" in Swahili — and this project is about protecting it.
This initiative was born from a conversation between two women: one from Mozambique and one from China, both committed to building solutions for those who need them most. In Mozambique, the numbers are staggering:
- 14 million malaria cases annually
- 33% hypertension rate, mostly undiagnosed until a stroke occurs
- Rural doctor-to-patient ratio: 1 : 10,000
Behind these statistics are real people.
My grandmother waited three days for a clinic appointment for what turned out to be severe malaria. Neighbors have suffered strokes because they didn’t know their “headache” was a hypertensive crisis.
People don’t gamble with their health because they don’t care — they gamble because they are trapped in the deadly gap between symptom onset and access to medical care.
When we saw the multimodal capabilities of Gemini, we asked:
“What if every smartphone in Mozambique could perform medical triage?”
Not to replace doctors — but to ensure people reach them at the right time, with the right urgency.
What Mpingo Does
Mpingo is a multimodal AI triage system that performs early medical screening and risk stratification for Mozambique's deadliest conditions:
- Malaria
- Diarrheal diseases
- Cholera
- Hypertension
User Flow (3 minutes total)
1. Multimodal Input
- Voice — Describe symptoms in Portuguese
- Photo — Rashes, BP readings, swelling
- Text — Duration + medical history
2. AI Analysis
- Gemini processes voice transcription + image + text
- Applies WHO IMCI guidelines + Mozambique disease patterns
- Generates clinical risk score
3. Risk Stratification
| Level | Meaning |
|---|---|
| 🔴 EMERGENCY | Immediate care required |
| 🟡 URGENT | Clinic visit within 24h |
| 🟢 MONITOR | Safe self-care + escalation rules |
4. Care Guidance
- Where to go
- What tests to request
- Danger signs to watch
- Nearest clinics (Google Maps grounding)
Core Principles & Key Learnings
Technology Must Meet People Where They Are
Innovation in emerging markets means accessible integration, not cutting-edge hardware.
- Low smartphone penetration
- Majority devices are Android
- Alignment with Google ecosystem
The Power of Longitudinal Context
The breakthrough is temporal reasoning, not single symptom detection.
Gemini’s long context allows years of history to be weighed against acute symptoms, creating:
Medical order out of chaos
Resilience as an Ethical Choice
Offline-first encryption = ethical obligation.
Medical records remain:
- Local
- Secure
- Available without internet
How We Built It
Team Approach
Three perspectives combined:
- Local ground truth
- Technical AI architecture
- Clinical validation
Technical Stack
Frontend
- React 19
- Vite
- Progressive Web App (PWA)
- Designed for 16GB entry devices
AI Pipeline
| Model | Purpose |
|---|---|
| Gemini 3 Pro Preview | Assist (clinical support) + Epidemic (disease analysis); advanced medical reasoning, multi-record comparison, pregnancy risk safety evaluation |
| Gemini 3 Flash Preview | Archive (documents & media analysis) + BMI insights; fast OCR, speech-to-text, and general health guidance |
| Gemini 2.5 Flash | Pharmacy (drugstore & map search); supports Google Maps grounding tools |
UI & Design System
- Tailwind CSS
- Material 3 Design Tokens (M3 styling system)
Routing
- React Router 7 (HashRouter for PWA navigation)
Browser & Device APIs
- MediaDevices API (camera + microphone input for document and voice capture)
- Geolocation API (location-aware pharmacy search)
Internationalization (i18n)
- Native English (EN) / Portuguese (PT) dual-language support
- Context-based language switching
Core SDK
- Google GenAI SDK (
@google/genai)
Data Security
- Local Base64 obfuscation
- 100% records remain on-device
Clinical Risk Algorithm
Risk Score Calculation
$$ \mathrm{Risk\ Score} = \sum_{i=1}^{n} w_i \cdot f_i $$
Where
$$ f_i = \text{clinical features} $$
$$ w_i = \text{weights adapted for Mozambique} $$
Risk Classification
$$ \mathrm{Classification} = \begin{cases} \mathrm{Emergency\ (RED)} & \text{if } Score \ge 7 \ \mathrm{Urgent\ (YELLOW)} & \text{if } 4 \le Score < 7 \ \mathrm{Monitor\ (GREEN)} & \text{if } Score < 4 \end{cases} $$
Key Challenges & Solutions
Paper-to-Digital Trust Barrier
Solution: Gemini Vision extracts meaning from handwritten notes and X-rays.
Medical Accuracy vs AI Uncertainty
Hard-coded safety overrides:
$$ \textbf{If } BP > 180/120 \;\; \textbf{or} \;\; O_2 < 92\% \;\; \Rightarrow \;\; \textbf{Automatic RED Alert} $$
Building for 2G Reality
Optimized to 500KB/session
- Client-side compression
- Streaming responses
Low Image Quality
AI switches to verbal interview when images are insufficient.
What We're Proud Of
- Democratized specialist triage
- Clinical equity (voice + visual access)
- Offline resilience
What’s Next
Next 3 Months
- Clinical validation study
- Pilot in 3 provinces
- Health worker training
6-Month Vision
- 10,000+ triage sessions
- Algorithm refinement
- USSD deployment with telecoms
Long-Term
- Mozambique’s early-warning health layer
- Southern Africa scaling
- Published clinical outcomes
Success Metric
If Mpingo prevents even 100 deaths in its first year — it will have succeeded beyond our wildest dreams.
Built With
- css
- css3
- geminiapi
- google-cloud
- google-generative-ai
- google-maps
- html5
- pwa
- react
- tailwind
- typescript
- vite
- web-rtc


Log in or sign up for Devpost to join the conversation.