Vo | Devpost

Introduction
Landing
Patient Dashboard

Inspiration

Physician burnout is one of the biggest challenges in healthcare today. A major contributor is the overwhelming administrative burden: doctors spend up to two hours on paperwork for every one hour of patient care. Much of this “pajama time”—finishing charts, notes, and billing codes late at night—reduces quality of life and patient care.

Electronic Medical Records (EMR/EHR) systems, while essential, are often clunky and unintuitive, requiring extensive manual data entry. Instead of solving the problem, they make it worse. We wanted to build something that truly helps clinicians focus on their patients, not their paperwork.

What it does

ScribeAgent AI is a clean, intuitive web application that acts as a doctor’s personal AI scribe.

Record: After seeing a patient, Dr. Chen opens the ScribeAgent dashboard and clicks Record. She dictates a natural summary:
“Patient is a 34-year-old male presenting with a sore throat, fever, and swollen lymph nodes. Rapid strep test was positive. Diagnosis is acute streptococcal pharyngitis. I’m prescribing Amoxicillin 500mg, twice daily for 10 days, and ordering a follow-up throat culture.”
Process: She clicks Finish. A live status indicator walks her through the steps:
- Transcribing Audio...
- Extracting Clinical Entities...
- Querying Billing Database...
- Generating Documents...
Review: Within seconds, the dashboard populates with:
- ✅ A structured SOAP Note
- ✅ A suggested ICD-10 Billing Code: J02.0 – Streptococcal pharyngitis
- ✅ A pre-filled Prescription Order for Amoxicillin
- ✅ A drafted Lab Order for a throat culture
Approve: Dr. Chen reviews the generated documentation. Everything looks good. With one click on Approve & Sign, the data is pushed into a mock EMR system.

Result: Instead of 15 minutes of typing, Dr. Chen spends <1 minute approving structured documents.

How we built it

Frontend

Framework: Next.js (React) with TypeScript
Styling: Tailwind CSS + Shadcn/UI for fast, professional components
Features:
- Real-time status updates via WebSockets
- Clean dashboard for SOAP notes, billing codes, and orders
Deployment: Vercel

Backend (Agent Core)

Framework: Python with FastAPI (async, lightweight, hackathon-friendly)
Agent Orchestration: Mastra Agent Framework to manage workflows and tool calls
AI Tools:
- Speech-to-Text: OpenAI Whisper API for accurate transcription
- LLM (GPT-4):
- Entity Extraction (diagnosis, meds, labs)
- SOAP Note Generation
- Prescription & Lab Order drafting
- Billing Code Lookup: CSV-based ICD-10 lookup function for speed and offline reliability

Challenges we ran into

Designing a doctor-first interface that’s minimal yet powerful.
Making sure entity extraction from freeform speech works reliably across accents and phrasing.
Handling edge cases in clinical notes (abbreviations, shorthand, overlapping symptoms).
Integrating multiple AI steps into a seamless, real-time pipeline.

Accomplishments that we’re proud of

Built a working prototype that turns speech into structured clinical documents.
Designed a dashboard that is simple enough for doctors to use without training.
Achieved fast end-to-end processing, keeping the entire workflow under 10 seconds.
Created an extensible backend that can plug into real EMR/EHR systems in the future.

What we learned

How to combine multiple AI components (Whisper, GPT-4, custom lookups) into a single agent-driven workflow.
The importance of UI/UX in healthcare software—doctors need fewer clicks, not more features.
The potential of agent frameworks like Mastra for orchestrating multi-step AI tasks.

What’s next for ScribeAgent AI

Real EMR Integration: Connect with systems like Epic or Cerner via FHIR APIs.
Mobile App Support: Let doctors use the system directly from their phones.
Multi-language Support: Extend transcription and generation to Spanish and beyond.
Clinical Accuracy Tuning: Fine-tune entity extraction with domain-specific LLMs.
Security & Compliance: Add HIPAA-compliant authentication, encryption, and auditing.