Mpingo: Protecting Life through Multimodal AI Triage :contentReference[oaicite:0]{index=0}

Inspiration

Mpingo means "life" in Swahili — and this project is about protecting it.

This initiative was born from a conversation between two women: one from Mozambique and one from China, both committed to building solutions for those who need them most. In Mozambique, the numbers are staggering:

  • 14 million malaria cases annually
  • 33% hypertension rate, mostly undiagnosed until a stroke occurs
  • Rural doctor-to-patient ratio: 1 : 10,000

Behind these statistics are real people.

My grandmother waited three days for a clinic appointment for what turned out to be severe malaria. Neighbors have suffered strokes because they didn’t know their “headache” was a hypertensive crisis.

People don’t gamble with their health because they don’t care — they gamble because they are trapped in the deadly gap between symptom onset and access to medical care.

When we saw the multimodal capabilities of Gemini, we asked:

“What if every smartphone in Mozambique could perform medical triage?”

Not to replace doctors — but to ensure people reach them at the right time, with the right urgency.


What Mpingo Does

Mpingo is a multimodal AI triage system that performs early medical screening and risk stratification for Mozambique's deadliest conditions:

  • Malaria
  • Diarrheal diseases
  • Cholera
  • Hypertension

User Flow (3 minutes total)

1. Multimodal Input

  • Voice — Describe symptoms in Portuguese
  • Photo — Rashes, BP readings, swelling
  • Text — Duration + medical history

2. AI Analysis

  • Gemini processes voice transcription + image + text
  • Applies WHO IMCI guidelines + Mozambique disease patterns
  • Generates clinical risk score

3. Risk Stratification

Level Meaning
🔴 EMERGENCY Immediate care required
🟡 URGENT Clinic visit within 24h
🟢 MONITOR Safe self-care + escalation rules

4. Care Guidance

  • Where to go
  • What tests to request
  • Danger signs to watch
  • Nearest clinics (Google Maps grounding)

Core Principles & Key Learnings

Technology Must Meet People Where They Are

Innovation in emerging markets means accessible integration, not cutting-edge hardware.

  • Low smartphone penetration
  • Majority devices are Android
  • Alignment with Google ecosystem

The Power of Longitudinal Context

The breakthrough is temporal reasoning, not single symptom detection.

Gemini’s long context allows years of history to be weighed against acute symptoms, creating:

Medical order out of chaos


Resilience as an Ethical Choice

Offline-first encryption = ethical obligation.

Medical records remain:

  • Local
  • Secure
  • Available without internet

How We Built It

Team Approach

Three perspectives combined:

  1. Local ground truth
  2. Technical AI architecture
  3. Clinical validation

Technical Stack

Frontend

  • React 19
  • Vite
  • Progressive Web App (PWA)
  • Designed for 16GB entry devices

AI Pipeline

Model Purpose
Gemini 3 Pro Preview Assist (clinical support) + Epidemic (disease analysis); advanced medical reasoning, multi-record comparison, pregnancy risk safety evaluation
Gemini 3 Flash Preview Archive (documents & media analysis) + BMI insights; fast OCR, speech-to-text, and general health guidance
Gemini 2.5 Flash Pharmacy (drugstore & map search); supports Google Maps grounding tools

UI & Design System

  • Tailwind CSS
  • Material 3 Design Tokens (M3 styling system)

Routing

  • React Router 7 (HashRouter for PWA navigation)

Browser & Device APIs

  • MediaDevices API (camera + microphone input for document and voice capture)
  • Geolocation API (location-aware pharmacy search)

Internationalization (i18n)

  • Native English (EN) / Portuguese (PT) dual-language support
  • Context-based language switching

Core SDK

  • Google GenAI SDK (@google/genai)

Data Security

  • Local Base64 obfuscation
  • 100% records remain on-device

Clinical Risk Algorithm

Risk Score Calculation

$$ \mathrm{Risk\ Score} = \sum_{i=1}^{n} w_i \cdot f_i $$

Where

$$ f_i = \text{clinical features} $$

$$ w_i = \text{weights adapted for Mozambique} $$

Risk Classification

$$ \mathrm{Classification} = \begin{cases} \mathrm{Emergency\ (RED)} & \text{if } Score \ge 7 \ \mathrm{Urgent\ (YELLOW)} & \text{if } 4 \le Score < 7 \ \mathrm{Monitor\ (GREEN)} & \text{if } Score < 4 \end{cases} $$


Key Challenges & Solutions

Paper-to-Digital Trust Barrier

Solution: Gemini Vision extracts meaning from handwritten notes and X-rays.

Medical Accuracy vs AI Uncertainty

Hard-coded safety overrides:

$$ \textbf{If } BP > 180/120 \;\; \textbf{or} \;\; O_2 < 92\% \;\; \Rightarrow \;\; \textbf{Automatic RED Alert} $$

Building for 2G Reality

Optimized to 500KB/session

  • Client-side compression
  • Streaming responses

Low Image Quality

AI switches to verbal interview when images are insufficient.


What We're Proud Of

  • Democratized specialist triage
  • Clinical equity (voice + visual access)
  • Offline resilience

What’s Next

Next 3 Months

  • Clinical validation study
  • Pilot in 3 provinces
  • Health worker training

6-Month Vision

  • 10,000+ triage sessions
  • Algorithm refinement
  • USSD deployment with telecoms

Long-Term

  • Mozambique’s early-warning health layer
  • Southern Africa scaling
  • Published clinical outcomes

Success Metric

If Mpingo prevents even 100 deaths in its first year — it will have succeeded beyond our wildest dreams.

Built With

Share this project:

Updates