Vital Sync

Inspiration# Vital Sync - About the Project

Inspiration

Language creates walls. In medicine, those walls can be fatal. The inspiration for VitalSync came from a personal realization: medical tourism and diverse communities are growing, but our ability to communicate in critical care settings hasn't kept up. We saw patients struggling to describe pain using basic translation apps that missed nuance, and doctors frustrated by the inability to give precise instructions. We wanted to build something that wasn't just a translator, but a medical companion—one that understands "my chest feels heavy" is more than just a weight issue.

What it does

VitalSync is a real-time, bi-directional, medically-aware interpretation system.

Live Consultation: A doctor speaks in English, and the patient hears/reads it in Spanish (or vice versa) instantly using low-latency WebSockets.
Context Preservation: Unlike generic tools, VitalSync is prompted to maintain medical accuracy.
Smart Summarization: At the end of a session, it automatically generates a structured clinical note (SOAP format: Subjective, Objective, Assessment, Plan) for the doctor's records.

How we built it

We built VitalSync with a focus on speed and reliability.

Frontend: We used React with TypeScript and TailwindCSS for a clean, trust-inspiring UI. We implemented "optimistic UI" updates to make the chat feel instantaneous.
Backend: A robust FastAPI server manages the WebSocket connections. We utilized async python capabilities to handle multiple concurrent sessions without blocking.
AI & Logic: The core intelligence is powered by Google Gemini 2.0 Flash. We chose this model for its incredible speed-to-intelligence ratio.
Mathematical Concept: To ensure medical accuracy, we conceptualized our translation not as a direct mapping, but as a probability maximization problem where we maximize the probability of Clinical Intent ($C$) given the Input ($I$):

$$ P(C | I) = \frac{P(I | C) \cdot P(C)}{P(I)} $$

We tuned our system prompts to heavily weight Medical Terminology ($M$) within that context ($C$).

Challenges we ran into

The "Amnesia" Bug: We deployed to a serverless container platform and immediately noticed our sessions were vanishing. Every time we sent a message, the connection dropped. We realized we were using a local SQLite database on an ephemeral filesystem—literally wiping our memory every few seconds! We had to quick-pivot to designing a robust Postgres architecture for production adaptability.
WebSocket State: synchronizing state between the frontend (React) and backend (FastAPI) while handling real-time audio streams was tricky. Handling race conditions where a user speaks while the previous translation is still processing required careful state management.

Accomplishments that we're proud of

Latency: We achieved a near-conversational latency.
UX/UI: We built a "Dual-Persona" interface where the Doctor and Patient have distinct, accessible views that work on the same screen or separate devices.
Resilience: We built a graceful fallback mechanism. If the AI service hangs, the raw transcript is still delivered, ensuring communication never stops completely.

What we learned

Deployment matters: Code that runs on localhost is only 50% of the work. Understanding container lifecycles is crucial.
Prompt Engineering is distinct from Coding: Getting the AI to sound like a doctor and not a chatbot required iterative refinement of our system prompts.
The Power of WebSockets: REST APIs are great, but for a "human" feeling connection, distinct persistent connections are non-negotiable.

What's next for Vital Sync

EMR Integration: A one-click "Export to Epic/Cerner" button.
Multimodal Diagnostics: Allow patients to upload photos of visible symptoms (rashes, swelling) for the AI to analyze and describe to the doctor.
Offline Mode: A localized version for rural clinics with spotty internet coverage.