HealthVoice

Fresh app
Detects when 911 is needed
Detects 911 as well as suicide hotline

We were inspired by the idea of making medical information more accessible and less intimidating. In a healthcare setting, people often have questions but may not know how to describe their symptoms, where to find trustworthy information, or when a situation could require urgent attention. We wanted to explore whether a natural, voice-based conversation could make that first step easier while still grounding responses in reputable medical sources.

We built HealthVoice, a voice-first medical assistant using Gemini Live for real-time speech-to-speech interaction. The frontend was built with React, TypeScript, and Vite, with a phone-call-style interface that lets users speak naturally and see the conversation as a live transcript. On the backend, we added a medical retrieval component that searches MedlinePlus so that responses can be grounded in external medical information rather than relying entirely on the language model's internal knowledge. We also implemented a safety layer designed to identify potentially urgent situations and encourage appropriate emergency care when necessary.

One of the biggest things we learned was that building an AI application is much more than connecting a model to a UI. We had to think about real-time audio streaming, browser permissions, backend security, API authentication, retrieval, safety handling, and deployment as one system. We also learned how important it is to keep permanent API credentials on the server rather than exposing them in a public frontend.

The biggest challenges were getting real-time voice communication working reliably in the browser, handling audio processing and streaming, connecting the frontend to a deployed backend, and configuring CORS and environment variables correctly. We also had to be careful about the limitations of medical AI. Our system is designed to provide information and help users navigate questions—not to diagnose conditions, prescribe treatment, or replace a healthcare professional.

Ultimately, this project taught us how to combine generative AI, retrieval-augmented generation, real-time voice interaction, and safety-focused engineering into a single healthcare experience under a very limited development timeframe.