HealthBridge AI

A multilingual AI healthcare assistant powered by Amazon Nova that analyzes symptoms via voice and images, performs medical triage, and connects rural patients to the nearest healthcare facility.

Comment

Inspiration

Millions of people living in rural areas lack immediate access to healthcare professionals.
Often the nearest clinic is hours away, and many patients delay seeking medical help because they are unsure whether their symptoms are serious.

During research, we found that early symptom triage can significantly reduce preventable health complications, but rural communities frequently lack the tools to access this guidance quickly.

We built HealthBridge AI to address this gap.

Our goal was to create a voice-first AI healthcare assistant that allows patients to simply describe their symptoms in their own language, receive an AI-powered triage assessment, and get directed to the nearest healthcare facility.

By combining voice AI, multimodal analysis, and medical reasoning, HealthBridge AI can help communities access faster health guidance while also assisting doctors and public health officials.

What it does

HealthBridge AI is a real-time AI healthcare triage system designed for rural communities.

Patients can:

  • Describe symptoms using natural speech
  • Upload images of visible symptoms (rash, wounds, swelling)
  • Receive AI-powered triage guidance
  • Get routed to the nearest Primary Health Centre (PHC)

The system then:

  1. Converts speech to text using AI voice processing.
  2. Analyzes symptoms and images using multimodal AI.
  3. Performs medical reasoning using WHO guideline knowledge retrieval.
  4. Calculates urgency and recommended next steps.
  5. Finds the nearest healthcare facility using geospatial search.
  6. Sends emergency alerts when critical symptoms are detected.

Doctors also benefit from:

  • AI-generated case summaries
  • Explainable reasoning traces
  • outbreak detection insights

This creates a bridge between rural patients and healthcare systems.

How we built it

HealthBridge AI integrates several technologies to create a full AI healthcare pipeline.

Frontend

  • Next.js
  • TailwindCSS
  • Framer Motion
  • Web Speech API

Backend

  • FastAPI
  • WebSockets for real-time voice streaming
  • Background task processing

AI Stack

We used Amazon Nova foundation models to power the system:

  • Nova Sonic – real-time voice interaction
  • Nova Multimodal – medical image analysis
  • Nova 2 Lite – reasoning and triage
  • Nova Act – workflow automation

Data Layer

  • Supabase PostgreSQL
  • pgvector for AI embeddings
  • PostGIS for geospatial healthcare routing

Infrastructure

  • Redis caching (Upstash)
  • Twilio SMS alerts
  • Railway deployment
  • Vercel frontend hosting

The AI pipeline works as follows:

Patient Voice → Speech AI → Symptom Extraction
Optional Image → Vision AI Analysis
Symptoms + Image → Reasoning Model + WHO RAG
Urgency Score → PHC Location Search → Patient Guidance

Challenges we ran into

Building HealthBridge AI required solving several technical and design challenges.

1. Multimodal AI coordination

We had to combine voice, text, and image inputs into a single reasoning pipeline while ensuring the system remained responsive.

2. Real-time voice streaming

Implementing real-time speech processing required WebSocket communication between the frontend and backend.

3. Reliable medical reasoning

To improve accuracy, we implemented a RAG system using WHO medical guidelines so the AI could reference trusted medical sources.

4. Safety and responsible AI

Healthcare AI must be cautious.
We added:

  • emergency keyword guardrails
  • explainability traces
  • doctor override functionality
  • audit logging

5. Rural connectivity constraints

Many rural regions have slow internet.
We designed the system to be lightweight and optimized for low-bandwidth environments.

What we learned

This project taught us several important lessons:

  • AI systems are most powerful when they combine multiple modalities (voice + vision + reasoning).
  • Responsible AI is critical in healthcare applications.
  • Explainability helps doctors trust AI recommendations.
  • AI can play a major role in public health monitoring, not just individual diagnosis.

Most importantly, we learned that technology can significantly reduce healthcare access gaps when designed thoughtfully.

Impact

HealthBridge AI has the potential to:

  • Reduce delays in medical consultation
  • Improve early detection of health conditions
  • Support frontline healthcare workers
  • Provide healthcare guidance in underserved areas
  • Detect public health trends through anonymized analytics

Even simple triage guidance can help patients seek medical help sooner and avoid complications.

What's next for HealthBridge AI

Future improvements could include:

  • multilingual voice models for regional languages
  • integration with telemedicine platforms
  • electronic health record integration
  • mobile-first offline support
  • expanded outbreak monitoring dashboards

Our long-term vision is to create a global AI-powered health triage platform accessible to everyone, regardless of location.

Built With

Share this project:

Updates