MedMimic

Starting Page
Login Page
Briefing Page
Encounter Page
Debrief Page
Compare Page

Inspiration

In med school and nursing school, students practice psychiatric conversations with hired actors called "standardized patients." The problem is that hiring actors is expensive, so students rarely get enough practice. Even worse, if you say the wrong thing in a practice session, you cannot just pause, rewind 30 seconds, and try a different approach to see how the patient would have reacted.

We wanted to build a tool where healthcare students can practice talking to patients out loud, make mistakes safely, and rewind the conversation to see how one better question can change everything.

What it does

MedMimic is a clinical interview simulator where you play the clinician and talk to an AI patient, either with your voice in real time or by typing.

Meet the patient: Our main case is Jordan Lee, a 34-year-old who comes into a walk-in clinic asking for sleeping pills. Underneath the insomnia, Jordan recently lost his job, is dealing with severe depression, and has been saving pills at home.
Generate new patients on the fly: Users can also request new AI-generated patients with different conditions and personalities. For example, some patients walk in already angry and confrontational, while others are panicked, stoic, or suspicious.
Trust and emotional reactions: Every time you speak, MedMimic checks how you talked to the patient. If you show empathy and reflect their words, their trust goes up and they open up more. If you rush them, lecture them, or brush them off, their trust drops, and they visibly and audibly get angrier or shut down.
Checkpoints and Rewind: When the patient drops an important hint, MedMimic saves a checkpoint. After finishing the visit, you can rewind right back to that moment, ask a better follow-up question, and compare your two runs side by side.
Scored Debrief: At the end of the visit, you get a score out of 100 and feedback backed by real clinical interviewing guidelines (like C-SSRS and SAFE-T), showing what you missed and what you could have said instead.

How we built it

We built MedMimic using Next.js, TypeScript, Tailwind CSS, Supabase, and the Google Gemini API:

Gemini Live API (gemini-3.8-live) handles the real-time voice conversation so you can actually talk to the patient with your mic and hear them talk back with different voices and tones.
Gemini 3.5 Flash (gemini-3.5-flash) classifies what the student said on every turn (empathy, reflection, asking about mood, asking about suicide directly, or being dismissive) using structured JSON output. It also generates new patient scenarios when requested.
A custom state engine tracks the patient's trust score from 0 to 100 and controls which facts the patient is allowed to share. Instead of giving the AI the whole backstory at once, where it might accidentally spill everything on turn one, we only put facts into the prompt after the student earns enough trust and asks the right questions.
Gemini Embeddings (gemini-embedding-001) and gemini-3.5-flash-lite power the debrief page, matching the moments from your interview to clinical guidelines and writing specific feedback on your run.
Supabase handles user login and saving session runs in the cloud.

Challenges we ran into

The AI forgetting who the doctor was: Early on, if I said something blunt like "That doesn't sound very healthy," Gemini would break character and give me a generic "This is not medical advice, please see a doctor" message, even though the user was supposed to be the doctor. We fixed this by always starting the chat history with a clinician greeting so the model never lost track of its role, plus a filter that catches and rewrites any accidental disclaimers.
Randomly switching to Spanish: In voice mode, certain phrases and background mic noise sometimes made the speech recognizer think we were speaking Spanish, and the patient would reply in Spanish. Locking the audio transcription config to English fixed it.
The patient talking to itself: When using laptop speakers, the mic would sometimes pick up the patient's own voice and trigger another reply before we even said anything. We added an audio gate right after playback finishes and lowered the voice activity sensitivity so it only replies when we actually speak.
Rate limits and model speed: gemini-3.8-flash had a tight 20-requests-per-day free limit and was too slow for every single turn, so we switched our turn classifier and text replies to gemini-3.5-flash with fallbacks to flash-lite.

Accomplishments that we're proud of

Getting real-time voice, live lip-syncing on our custom SVG avatars, and our trust engine to all work together smoothly.
Making the patient's personality actually feel real. When you are dismissive and trust drops, the patient genuinely sounds frustrated and pushes back instead of acting like a polite chatbot.
The Rewind and Compare flow, where you can clearly see how changing a single sentence at a critical moment leads to uncovering suicide risk instead of sending the patient home with sleeping pills.

What we learned

Don't rely on prompts alone for game rules: Trying to tell an LLM "don't mention X until the user does Y" in one big prompt is super unreliable. Keeping locked facts completely out of the prompt until our code unlocks them worked 100% of the time.
Steering voice tone mid-call: We learned how to send silent director cues to Gemini Live in the middle of a WebSocket session so the patient's voice gets sharper when angry or softer when opening up.