MedTranslate

This is the dashboard, featuring Highlights (key details, allergies, etc.), Visits, Recording, and Medical Chat Assistant.
Export health reports.
Example health report.
AI Interpretation (transcript in next screenshot)
Here's the transcript.
Action Plan

Inspiration

Doctor visits can feel overwhelming, especially for older adults and anyone without a medical background, and many people leave appointments forgetting key details or confused by clinical jargon. We built MedTranslate to reduce that gap by helping patients understand what is being said in real time while quietly creating a useful personal record from each visit, all with a low-friction, private-first experience.

What it does

MedTranslate listens during a visit, converts medical language into plain English, and presents a live explanation with key terms, suggested follow-up questions, and practical next steps. After the visit ends, it generates a structured recap and extracts important health facts such as medications, allergies, conditions, and procedures into a longitudinal record the patient can browse later alongside past visits and contacts, with a grounded Q&A experience that answers questions based on what is already in their saved history.

How we built it

We built MedTranslate as a local-first web app using React, TypeScript, and Vite on the frontend, with Node.js and Express on the backend, Dexie/IndexedDB for on-device persistence, and Gemini for structured summarization, recap generation, extraction, and record-grounded Q&A. The app streams transcript chunks to the backend, transforms them into strict JSON outputs, sends live updates back through SSE, and saves both visit-level artifacts and extracted profile facts when a session ends, while PWA packaging keeps the demo installable and app-like.

Challenges we ran into

Our biggest challenges were balancing speed and quality for real-time updates, keeping model output reliably structured under live conditions, and designing an interface that remains readable and calm under cognitive load. We also had to handle imperfect speech recognition input, maintain continuity across streaming chunks, and preserve a polished demo experience while integrating multiple moving parts across frontend state, backend session handling, and local storage.

Accomplishments that we're proud of

We’re proud that we shipped a fully working end-to-end MVP that can be demoed quickly and clearly: users can start a visit, see live plain-language interpretation, end the session, and immediately view a saved recap with extracted health data reflected across the app. We’re also proud of the product feel, especially the clear information hierarchy and local persistence model that keeps the experience practical and resilient for demos.

What we learned

We learned that in healthcare-adjacent experiences, trust comes from clarity, consistency, and grounding more than raw model sophistication. We also learned how important schema discipline and defensive parsing are for stable AI UX, how effective local-first persistence can be for privacy-sensitive workflows, and how much thoughtful interface design influences whether people actually feel supported by AI in stressful moments.

What's next for MedTranslate

Next, we want to improve extraction quality controls and deduplication, strengthen transcription reliability, and expand the personal record experience with richer editing and review flows so users can confirm or correct auto-captured facts. We also plan to make grounded Q&A more explicit about source provenance, improve offline and accessibility behavior, and harden reliability so MedTranslate can move from hackathon MVP toward real-world readiness.