Verba

Verba. Every word, a lifeline.
Problem: 80% of serious medical errors occur during patient handoffs.
Verba provides real-time, structured, and reliable incident reporting—by voice.
Verba Architecture
Verba Frontend

Inspiration

Every second counts: the difference between life and death. Yet in today’s EMS-to-hospital handoffs, an estimated 80% of serious medical errors occur during patient handoffs due to vital patient information being too often delayed, distorted, or completely lost.

During high-stress calls—late at night, mid-transport, or while multitasking—first responders can't always recall or relay every detail, especially without reliable tools to support them. This puts thousands of lives at risk every year.

So we built Verba: A voice-first AI agent built to streamline the EMS-to-doctor handoff—accurately, instantly, and effortlessly.

What it does

Verba is a real-time voice agent that ensures nothing gets lost in translation during the most critical moments of emergency responses. Designed for EMS-to-hospital handoffs, Verba listens to paramedics’ verbal reports on scene and automatically transcribes, extracts, and structures all key medical data using a voice-first, AI-assisted workflow. It’s trained to be as efficient as possible, asking clarifying questions only when absolutely necessary.

Verba then uses advanced custom agents to deliver structured reports that directly integrate with existing electronic patient care report (ePCR) systems. These reports appear instantly on the Doctor Dashboard prioritized on criticality, reducing time-to-treatment and minimizing human error.

How we built it

Voice Agent:

Vapi: Built and deployed an advanced AI voice agent. Handles natural conversations in real-time, makes and receives calls, and is built to seamlessly connect with major ePCR/EHR systems. Designed to respond only when necessary to save as much time as possible.
Claude: Used for advanced language understanding and summarization. Aids in refining handoff notes into concise, medically actionable key terms and summaries for rapid physician review.

Frontend:

React application: Real-time updates. Provides a clean, clinician-friendly UI.
Leaflet Maps API: Integrated live mapping to show real-time EMS locations and scene data directly on the Doctor Dashboard. Helps doctors anticipate patient arrival and triage in advance.

Backend:

Supabase: Used as a scalable Postgres database with built-in authentication and row-level security. Enabled real-time syncing of structured patient data and secure access control for doctors and EMS teams.
Medical database: Thousands of EMS-specific terminology, abbreviations, and field-data.
Letta: A custom medical language model agent, based on GPT-4o mini and trained on medical database. Parses transcriptions from Vapi into structured, clinically relevant data fields for doctor review. Seamlessly connects with existing major ePCR/EHR systems like Zoll and ESO.

Challenges we ran into

Balancing tradeoffs between real-time responsiveness and accuracy
Acquiring an extensive database that would power the report generation and semantic understanding of the analyzed speech
Addressing variations in real-world speech, including background noise and enunciation
Unifying multiple AI agent frameworks into one cohesive product

Accomplishments that we're proud of

Created a fully functional AI agent for medical emergencies in 24 hours
Minimized latency in the voice call and dashboard updates to be near instant
Built flexible report-generation pipeline that can be adapted for various medical emergencies and beyond
Ensured the agent is accessible via phone

What we learned

How to integrate multiple modern APIs in one cohesive project
EMS reports are unstandardized and incomplete
Instant verbal feedback is helpful for EMS first responders to ensure all necessary information about an emergency is recorded

What's next for Verba

Multimodal integration: live video feeds—especially body cam footage—to enhance situational understanding through real-time visual segmentation and tracking
Security: HIPAA-compliance and full encryption (including voice data in transit) will be prioritized to protect patient privacy and system integrity
Model base: Increasing model complexity while maintaining quick speed
Integration: Further integration of EMS Patient Care Report with hospital workflow