LifeLine

You shouldn't have to be put on hold by 911.

Comment

Inspiration

Emergency 911 dispatch centers are critically understaffed nationwide. During peak hours or mass-casualty events, callers are frequently put on hold — sometimes for over 5 minutes. In a severe trauma situation, a 5-minute hold time is deadly.

The current system also relies on a slow, sequential "telephone game" where data is manually relayed from caller \(\rightarrow\) operator \(\rightarrow\) dispatcher \(\rightarrow\) ambulance \(\rightarrow\) hospital, with critical details lost or delayed at every handoff. We realized Conversational AI could act as an instant overflow safety net and a parallel data-router that fans information out to every downstream party the moment it's spoken.

What it does

LifeLine is an autonomous, AI-powered emergency dispatcher that catches 911 overflow calls when human operators are saturated. Instead of waiting on hold, a panicked caller is instantly connected to a calm, hyper-responsive AI dispatcher built on ElevenLabs Conversational AI.

As LifeLine speaks with the caller — calming them down, walking them through basic survival protocols, and gathering facts — it simultaneously runs structured entity extraction in the background. The instant it has enough context, it:

  1. Auto-dispatches EMS via an SMS packet (location, nature, victim count, symptoms, severity tag) sent straight to the responding paramedics.
  2. Generates a live ER pre-arrival prep card with triage level, suggested bay, prep items, and suspected differential diagnoses — pushed to the receiving hospital before the ambulance leaves.
  3. Renders a live mission-control dashboard for human supervisors: live transcript, geocoded incident map, ambulance routing animation along real streets, dispatch SMS history, and a continuously-updated plain-English call summary.
  4. Auto-ends the call once the conversation completes and reroutes any escalation back to a human dispatcher.

The dashboard is built around quiet, cinematic glass-morphism — Apple-Intelligence-style frosted panels floating in a single static beam of dim purple light. The aesthetic is deliberate: dispatchers are already overwhelmed; the UI breathes calmly and never demands attention.

How we built it

We built a real-time, event-driven voice architecture obsessed with end-to-end latency.

  • Telephony: Twilio Voice provisions the inbound number and pipes raw caller audio over a WebSocket Media Stream.
  • Voice Agent: ElevenLabs Conversational AI handles STT and ultra-low-latency neural TTS — chosen specifically for its emotional warmth, which measurably helps panicked callers.
  • Reasoning Brain: Google Gemini 2.5 Flash drives the agent's responses and simultaneously fires structured JSON tool calls (update_incident, trigger_dispatch, end_call) without ever interrupting the audio.
  • Geocoding: OpenStreetMap Nominatim turns the caller's spoken address into coordinates the moment it's mentioned — no Mapbox token, no API key, no credit card required.
  • Live Map: Leaflet + CartoDB Dark Matter tiles + OSRM driving directions. When dispatched, an ambulance icon spawns at a deterministic random nearby spot and animates a real road-following polyline through the pickup to the nearest hospital. The target destination is selected from 46 curated Texas Level-I trauma centers by calculating the central angle between points via the Haversine formula:

$$ \Delta\sigma = 2 \arcsin\left(\sqrt{\sin^2\left(\frac{\Delta\phi}{2}\right) + \cos(\phi_1)\cos(\phi_2)\sin^2\left(\frac{\Delta\lambda}{2}\right)}\right) $$

  • Outbound Messaging: Twilio SMS to EMS for dispatch packets and ER prep updates (with a WhatsApp Sandbox fallback we built to sidestep US A2P 10DLC carrier filtering during the demo).
  • Frontend: Next.js 14 + TypeScript dashboard with Server-Sent Events streaming live state deltas, Framer Motion micro-animations, and custom CSS-only glass panels with mask-composite gradient borders.
  • Backend: FastAPI Python, in-memory incident state behind an RLock, and async debouncing on Gemini calls so a flurry of field updates collapses into a single LLM invocation.

Challenges we ran into

  • API Stability: ElevenLabs Scribe Realtime dropping mid-call with insufficient_funds. We restructured task ownership so STT failure can't tear down the Twilio Media Stream, allowing the agent to gracefully degrade.
  • Carrier Filtering: US A2P 10DLC carrier filtering silently blocking demo SMS. We built a runtime MESSAGING_CHANNEL switch that routes through the Twilio WhatsApp Sandbox instead.
  • LLM Constraints: Gemini 2.5 Flash returning empty strings because the thinking-token budget was eating the entire output allotment. We pinned thinking_budget=0 for structured-JSON paths.
  • React Lifecycle Hurdles: React StrictMode + Leaflet double-mounting destroyed the map's pane references mid-render, crashing async OSRM addTo calls with cryptic appendChild errors. This required disabling StrictMode and rebuilding the lifecycle around refs that survive double-mounts.
  • Edge-Case Map Scaling: flyToBounds over transcontinental distances silently broke the Leaflet tile renderer when out-of-state callers hit our Texas hospital network. We now compute a synthetic fallback hospital for any pickup distance \( d > 250 \text{ miles} \) from the closest real one.
  • Animation Synchronization: A rAF clock skew where the requestAnimationFrame timestamp came in microseconds earlier than the performance.now() timestamp recorded right before scheduling it. This yielded a negative delta:

$$ \Delta t = t_{\text{rAF}} - t_{\text{perf}} < 0 $$

This resulted in an index calculation of i = -1 and caused a hard crash on the very first animation frame. We resolved this by clamping the delta baseline.

Accomplishments that we're proud of

What we're proudest of is that this project genuinely has the potential to do massive social good. AI agents catching overflow 911 calls could save lives at the bottleneck moment when seconds matter most.

Beyond the mission, we are incredibly proud of the technical execution: 108 backend tests passing, a free-tier map stack with zero API keys, real road-following ambulance routing, geocoded live caller tracking, and a UI design that steps outside the box of typical hackathon projects.

What we learned

Building LifeLine was a masterclass in streaming architecture: bidirectional WebSockets, raw telephony audio, prompt structuring for an LLM that has to talk and extract data in parallel, SSE state-delta broadcasting, and the surprising subtleties of getting a map library to play nicely with React's dev-mode double-mount.

We also gained a deep appreciation for the protocols real 911 dispatchers run every shift — every UX choice in the dashboard is a tribute to how much cognitive load they carry.

What's next for LifeLine

  • Direct CAD Integration: Build direct hooks into real Public Safety Answering Point (PSAP) dispatch software so LifeLine can hand a fully-formed incident to the human queue with one click.
  • Multi-language Support: Add Spanish, Mandarin, and Vietnamese coverage on day one.
  • Predictive Triage Routing: Route to receiving hospitals based on live emergency department wait times and trauma capacity, rather than just raw distance.
  • National Expansion: Expand to a full national hospital network beyond our curated Texas list.
  • Compliance & Auditing: Integrate tamper-evident call recording and auditability logs to meet the rigorous legal and quality-assurance requirements of real dispatch centers.

Built With

Share this project:

Updates