VocalDoc - AI Medical Assistant

AI-powered medical symptom analysis through voice. Speak your symptoms naturally and get instant AI analysis with potential conditions, severity assessment, and recommended next steps.

Comment

Inspiration

Healthcare accessibility remains a global challenge. Many people struggle to articulate their symptoms effectively to medical professionals or lack immediate access to guidance. We were inspired to create VocalDoc after realizing that voice-based interaction could bridge this gap — making symptom analysis more intuitive and accessible to everyone, regardless of their medical knowledge or writing abilities.

What it does

VocalDoc is an AI-powered medical assistant that transforms spoken symptom descriptions into actionable health insights. Users speak naturally about their symptoms, and our system provides:

  • Real-time voice transcription with contextual understanding
  • AI-powered symptom interpretation using OpenAI for natural language processing
  • Likely condition suggestions with confidence scores
  • Severity indicators (mild, moderate, severe)
  • Emergency flags for potentially critical symptoms
  • Personalized next steps and self-care suggestions
  • Accessible, intuitive interface designed for all users

⚠️ Disclaimer: VocalDoc is not a substitute for professional medical advice, diagnosis, or treatment. Always consult a licensed healthcare provider.

How we built it

Frontend Architecture

  • React 18 + TypeScript for modern, type-safe development
  • Tailwind CSS for responsive and elegant UI styling
  • Vite for ultra-fast development and production builds
  • Web Audio API for real-time audio recording and waveform rendering
  • MediaRecorder API for audio capture with format fallback support

Backend Infrastructure

  • ElysiaJS for high-performance, type-safe API development
  • Google Cloud Run for scalable, serverless deployment
  • RESTful API with secure authentication layers
  • OpenAI API to analyze symptom descriptions and extract medically relevant insights
  • Multi-format audio support (WebM, MP4, OGG, WAV)

Key Technical Features

  • Smooth real-time waveform animation while recording
  • Dynamic audio format detection and conversion
  • Fully responsive, mobile-first interface
  • Robust error handling and user feedback system
  • Production-ready deployment pipeline with CI/CD

Challenges we ran across

  1. Audio Format Compatibility: Different browsers support different audio formats. We built a dynamic detection layer with fallback support.
  2. Real-time Audio Processing: Maintaining performance while rendering waveforms required careful Web Audio API optimization.
  3. AI Interpretation Accuracy: Translating free-form voice input into meaningful symptom summaries demanded prompt engineering and responsible AI design.
  4. Cross-Browser Issues: MediaRecorder API inconsistencies across browsers required testing and polyfills.
  5. UI/UX Balance: Making the interface both trustworthy (medical) and accessible (approachable) was a delicate design challenge.

Accomplishments that we're proud of

  • Built a production-ready medical voice assistant in record time
  • Seamlessly integrated voice input with AI interpretation and actionable suggestions
  • Achieved real-time waveform visualization
  • Deployed a scalable backend on Google Cloud Run
  • Designed a fully accessible and mobile-responsive UI
  • Implemented responsible AI practices with clear disclaimers and user protections

What we learned

  • Advanced Web Audio API handling and waveform techniques
  • Building APIs with the ElysiaJS framework
  • Scalable deployment with Google Cloud Run
  • Navigating AI ethics in healthcare, including risk management and disclaimers
  • Browser quirks in audio recording and playback
  • Full-stack production deployment with CI/CD and monitoring

What's next for VocalDoc

  • Multi-language support for broader accessibility
  • Wearable device integration for continuous health tracking
  • Custom-trained medical AI models to improve condition matching
  • Doctor handoff integration for seamless escalation
  • iOS and Android app development
  • Telemedicine features for real-time consultations
  • Symptom history and health insights dashboard

Built With

Share this project:

Updates