VitalCoach.ai

Inspiration

Every day, our wearables generate thousands of data points—from our heart rate to our deep sleep stages to our blood oxygen levels. However, we realized that for the average person, this data is useless. It is just a bunch of fragmented numbers on a screen without any context. We wanted to bridge the gap between raw health tracking and actual medical intelligence by building an AI assistant that doesn't just show you your numbers, but actually tells you what to do about them.

What it does

VitalCoach is a real-time, multimodal AI health dashboard. It acts as your personalized medical copilot that continually monitors your health data. If your deep sleep is critically low and your HRV drops, the copilot will proactively warn you about your fatigue. Additionally, the dashboard features a multimodal chat interface: you can upload a photo of your lunch or record a voice note, and the AI will analyze the nutritional breakdown and how it will impact your glucose levels today.

How we built it

We utilized a powerful, modern tech stack powered by four incredible sponsors:

Nexla: We used Nexla as the core data pipeline to automatically intercept raw, unstructured Apple Health JSON data from our phones, clean it, and stream it securely to our backend.
DeepMind (Gemini): We integrated the gemini-2.0-flash-lite model as our AI reasoning engine. It's fast, multimodal, and capable of synthesizing text, health stats, and image analysis instantly.
Assistant UI: We built out the entire conversational frontend with Assistant UI, which allowed us to effortlessly craft a seamless chat interface with file upload and voice capabilities.
DigitalOcean: We containerized our entire application (React frontend and Node backend) using Docker and deployed it onto DigitalOcean's robust cloud infrastructure for maximum uptime and stability.

Challenges we ran into

One of our biggest hurdles was figuring out how to connect real-time iOS device data tracking to an AI model seamlessly. Working with continuous, messy health data required us to rely heavily on Nexla's reliable hooks to sanitize the information before Gemini could accurately draw conclusions from it.

Accomplishments that we're proud of

We are incredibly proud of successfully building a true multimodal chat experience that is fully integrated with a live data stream. Uploading a picture of a meal and having the AI instantly reference your live heart rate data to tell you how that exact food is going to affect you today felt like magic.

What we learned

We learned the incredible power of the Model Context Protocol (MCP). By giving the Gemini AI access to specific backend data-pulling tools, we were able to turn a generic language model into a highly specialized health expert that genuinely knew our daily metrics.

What's next for VitalCoach

We want to expand the data pipeline to support all major wearables (Garmin, Whoop, Oura), integrate a continuous glucose monitoring API, and allow the Copilot to proactively notify users via SMS if it spots a dangerous trend in their vitals.