Inspiration
Doctors often spend more time documenting patient interactions than actually interacting with patients. Clinical notes are usually written after consultations, leading to delays, missing details, and cognitive overload for clinicians.
We were inspired to build HealthScribe to reduce this documentation burden by assisting doctors during consultations — not by replacing their judgment, but by helping them capture conversations accurately and consistently in real time.
What it does
HealthScribe is an AI-powered clinical documentation assistant that converts doctor–patient conversations into structured, editable SOAP notes.
It:
Captures real-time doctor–patient conversations using voice recognition
Transcribes natural speech into structured SOAP-format clinical notes
Highlights important clinical signals such as:
Worsening symptoms
Sleep disturbance
Functional impairment
Emotional distress
Delayed care indicators
Flags contradictory patient statements for clinician review
Supports multilingual input (Hindi/English with auto-translation)
Allows doctors to fully review, edit, approve, or reject all generated notes
Shares summaries with patients or pharmacies only after doctor approval
HealthScribe assists documentation only and does not provide medical advice, diagnosis, or treatment decisions.
How we built it
Frontend: React 18 with Vite, JavaScript (ES6+), CSS3, and Lucide React icons
Browser APIs:
Web Speech API for real-time speech-to-text transcription
MediaRecorder API for audio capture
AI / NLP (Rule-based):
Language detection and translation
Symptom and timeline extraction using heuristics
Clinical signal highlighting
Consistency and contradiction checking
Risk prioritization through rule-based logic
The system is designed so that all outputs remain editable and under the clinician’s control.
Challenges we ran into
Handling unstructured, conversational medical speech accurately
Avoiding over-interpretation of symptoms while still highlighting relevant clinical signals
Managing multilingual speech (Hindi–English code-mixing)
Designing a system that is helpful without crossing into medical decision-making
Ensuring transparency and trust in AI-generated clinical notes
Accomplishments that we're proud of
Built a working real-time voice-to-SOAP documentation pipeline
Successfully highlighted clinically relevant signals without automated diagnosis
Implemented contradiction detection to assist clinician review
Maintained strict human-in-the-loop control for all outputs
Designed a healthcare-safe AI system with clear scope boundaries
What we learned
Clinical AI must prioritize safety, explainability, and human oversight
Even simple rule-based NLP can provide high clinical value when designed carefully
Clear scope definition is essential when building healthcare technology
Multilingual healthcare tools significantly improve accessibility
Documentation support alone can have a meaningful impact on clinician workload
What's next for HealthScribe
Improved speaker separation (doctor vs patient)
More robust clinical signal detection with expanded rule sets
Integration with EHR systems (documentation-only workflows)
Support for additional languages
Enhanced clinician feedback loops for continuous improvement
Optional on-device processing for better privacy
Log in or sign up for Devpost to join the conversation.