Inspiration

Physicians spend roughly two hours on documentation for every one hour spent with patients — a pattern often called "pajama time," since much of that charting happens after hours, at home. It's one of the most cited drivers of physician burnout, and it directly eats into the time doctors could spend with patients. Ambient clinical documentation is already a real, funded product category (Nuance DAX, Abridge, Ambience Healthcare), which told us this wasn't a toy problem — it was a real one we could take a genuine crack at in the time we had.

What it does

MediScribe AI listens to a doctor-patient consultation and turns it into a complete SOAP note in under 30 seconds, through three agents:

  1. Transcription Agent — converts the recorded audio into clean text using Groq's Whisper Large v3.
  2. Medical NLP Agent — extracts structured entities from the transcript: symptoms, medications, allergies, vitals, duration, and diagnosis, using GPT-OSS 120B.
  3. SOAP Generator + Safety Agent — produces a structured SOAP note (Subjective, Objective, Assessment, Plan) formatted to standard clinical conventions, and separately surfaces safety flags — missing critical information or potential red flags in the consultation — so they're never buried inside the note text.

How we built it

Next.js 14 with the App Router on the frontend, calling two Next.js API routes on the backend: one for transcription, one for the two-stage analysis (entity extraction → SOAP generation), both hitting the Groq API. Clinical notes are currently saved to the browser's localStorage, so the demo works end-to-end with zero external database dependency. Styling with Tailwind and shadcn/ui, animations with Framer Motion, deployed on Vercel.

Challenges we ran into

Getting the Medical NLP and SOAP agents to return clean, consistently-structured JSON from a free-text transcript took real prompt iteration — LLM output format can drift, and for a clinical note that's a real risk, not just a formatting annoyance. We also had to think carefully about audio handling given the sensitivity of the data: audio is processed in memory and never stored, and inputs are validated and sanitized before they ever reach the model.

Accomplishments that we're proud of

A working, deployed, end-to-end pipeline — audio in, structured note out — built around a real and well-documented clinical problem, with an explicit safety-flag step that most quick documentation-tool prototypes skip entirely.

What we learned

That the hard part of "AI for healthcare" isn't calling a model — it's the layer around the model: input validation, deciding what happens when the model's output doesn't match your schema, and being honest about what a demo can and can't claim about clinical accuracy.

What's next for MediScribe AI

  • Persistent, multi-user storage via a proper Postgres database (Neon/Render), with a dedicated backend for sync across devices.
  • Authentication, so notes are tied to a provider account rather than a single browser.
  • A real compliance path — this is a demo, not HIPAA-compliant, and a production version would need encryption at rest, audit logging, and compliant hosting before touching real patient data.
  • Schema-validated model output (e.g. via Zod) with retry logic, so the pipeline doesn't silently break if the model's response format drifts.## Inspiration

Built With

Share this project:

Updates

Submission history