Aidora: Real-Time Health Triage Agent

Architecture Diagram
Simple view
Aidora
UI

🌐 Explore Aidora: https://frontend-pearl-iota-80.vercel.app/webiste 🎥 Watch Live Demo: https://www.youtube.com/watch?v=CniiS-kSj9c 📄 Full Docs: https://aidora-doc-9f7q.vercel.app/

Inspiration

It started with a moment I couldn't forget.

During a journey, I witnessed a young girl suddenly become very unwell. Her condition deteriorated within minutes she struggled to stand, and soon she could not speak clearly. Everyone around her wanted to help, but no one knew what to do. There was no doctor nearby, no clinic, no immediate guidance.

Every second felt heavier than the last. By the time help finally arrived, her condition had already worsened significantly.

That night I couldn't sleep. I kept thinking how many people face this exact situation every single day? A medical emergency, and zero immediate guidance.

That one moment became the entire reason Aidora exists.

What It Does

Aidora is a real-time, multimodal AI health triage assistant that provides immediate guidance during uncertain medical situations.

Visual Observation continuously analyzes visible symptoms through camera in real time.

Voice Interaction natural spoken conversation, asking follow-up questions like a real triage nurse.

Contextual Reasoning connects everything it sees and hears to deliver a clear urgency assessment.

Aidora performs triage only it evaluates urgency and guides next steps. It does not diagnose diseases and does not replace professional medical care.

How We Built It

Aidora was built on Google Gemini Live API combining live audio streaming, real-time vision, and contextual memory in a single unified session.

Live Audio Pipeline microphone input streamed continuously with native turn-taking support.

Real-Time Camera Feed camera frames injected directly into the live session alongside audio.

Triage Reasoning Layer carefully structured system prompt shapes urgency classification through three phases:

Observation Phase Aidora begins watching and listening. Assessment Phase targeted follow-up questions based on what it sees and hears. Triage Phase clear urgency classification with safe next-step guidance.

📄 Full Docs: https://aidora-doc-9f7q.vercel.app/

Challenges We Ran Into

Responsible AI boundaries designing safe triage guidance without crossing into diagnosis territory required careful prompt design and many iterations.

Multimodal synchronization balancing camera frame frequency with audio streaming without overwhelming the session.

Real-world reliability consistent performance across varied lighting, accents, and symptom presentations.

Accomplishments We're Proud Of

Turned a personal experience into a working real-world solution.

Built a genuinely live multimodal system one model seeing and hearing simultaneously.

Designed with safety and transparency at every step.

Delivered a complete functional triage agent within hackathon constraints.

What We Learned

Multimodal is a different category of product vision and voice together enables reasoning impossible with either alone.

System prompt design is serious engineering Aidora's triage quality was almost entirely determined by how carefully we structured its instructions.

The best technology solves problems that keep people up at night. This one kept us up for a very long time before we built it.

What's Next for Aidora

Multilingual support health emergencies don't happen only in English.

Offline-first architecture for areas with limited connectivity.

Emergency contact integration share triage assessment with family or emergency services.

Age-specific triage protocols urgency criteria differ across age groups.

Healthcare platform integration connecting triage output to local clinic systems.

Aidora is not meant to replace doctors. It is an intelligent first-response assistant for the moments when doctors aren't immediately reachable. Because sometimes, the most important thing isn't a diagnosis it's knowing whether to wait, act, or call for help right now. 🩺