Halo AI

Inspiration Omar on our team recently experienced a family emergency where the ambulance was delayed and the wrong responders arrived. His father recovered, but it exposed deep flaws in how 911 calls are triaged. We learned this isn’t rare. The average U.S. ambulance response time ranges from 7–14 minutes, and many calls are misunderstood or misprioritized. We wanted to see if real-time AI could help.

What It Does Transcribes live emergency calls using VAPI Uses AI agents to analyze transcript in real time Ranks each call’s urgency from 1–10 Identifies key concern tags like “domestic violence” or “injury” Streams all insights live to a dispatcher dashboard

How We Built It Frontend: React with WebSocket updates Backend: FastAPI for VAPI webhook + AI API endpoints Voice: VAPI.ai for real-time transcription AI: Claude Sonnet 4 for scoring and concern detection Orchestration: Orkes Conductor to trigger workflows based on incoming data

Challenges We Ran Into Simulating a call that felt real-time Managing fast data streams and async updates Fine-tuning Claude prompts for safety-critical edge cases Making dual-speaker transcription work clearly and accurately

Accomplishments That We're Proud Of Fully working demo with transcript, urgency score, and concern detection Integrated Orkes and Claude workflows successfully Fast, reliable data flow between phone, backend, and UI Built a real-time system that could meaningfully assist human dispatchers

What We Learned Real-time AI triage is possible with thoughtful prompting and workflow design Orkes Conductor makes asynchronous task management intuitive Voice input combined with LLMs unlocks a powerful new modality Team collaboration under a tight deadline is key to fast iteration

What’s Next for Halo AI Add multilingual call support Integrate caller history and GPS data Expand concern detection to cover mental health and self-harm Partner with emergency systems and healthcare responders Build a dispatcher training tool using simulated calls