Inspiration
Elfie's Medical Scribe challenge stood out because it solves a painful, high-frequency problem with clear evaluation criteria. Doctors and clinical staff lose significant time to documentation, and that time does not create direct patient value. We were inspired to build something narrow, useful, and clinically structured instead of trying to build a full healthcare platform in a week.
What it does
ClinicScribe AI takes a consultation audio recording and produces three outputs:
- a transcript
- structured clinical information, including symptoms, findings, diagnosis, and treatment
- a doctor-facing report plus a simpler patient-friendly summary
The MVP is designed for multilingual consultations and mixed-quality recordings. It aims to help clinicians review a visit faster, reduce note-writing burden, and give patients clearer next steps after the consultation.
How we built it
We built the system as a staged pipeline rather than a single prompt. First, the audio is transcribed. Then the transcript is segmented into clinically meaningful chunks. Qwen is used to extract structured medical content and generate two separate summaries: one for clinicians and one for patients. A validation layer checks for missing core sections such as reason for visit, relevant history, findings, diagnosis, and treatment plan.
The prototype includes a simple upload-and-review workflow:
- upload consultation audio
- generate transcript
- extract structured clinical data
- render doctor report and patient summary
- allow quick review and correction before export
Challenges we ran into
The hardest challenge was not transcription alone; it was preserving clinical meaning across messy real-world speech. Consultations are not neat. People interrupt each other, switch languages, and mention key symptoms indirectly. Another challenge was making the patient summary clearer and safer without over-claiming or losing important context.
Accomplishments that we're proud of
We're proud that the product is focused, clinically structured, and directly aligned with Elfie's evaluation criteria: correctness, multilingual handling, completeness, actionability, and UX clarity. We also separated the outputs by audience, because doctors and patients do not need the same wording or level of detail.
What we learned
We learned that healthcare AI becomes much stronger when the workflow is decomposed into reliable steps instead of asking one giant model call to do everything. We also learned that good UX in healthcare means helping the user verify, not just generate. Reviewability matters as much as model quality.
What's next for ClinicScribe AI
Next, we want to improve multilingual robustness, add better speaker separation, and benchmark the system on more unseen consultation recordings. We also want to expand the review layer so clinicians can confirm, edit, and approve notes quickly before sending them to downstream systems.
Log in or sign up for Devpost to join the conversation.