VoiceVitals

ML web app detects cognitive decline/disabilities(Dementia) via speech analysis—Automate screening in clinics for fast, non-invasive early detection and monitoring. Scalable for Research Funding

Comment

Inspiration

Families caring for loved ones with neurodegenerative conditions say daily symptom/constant clinic visits tracking feels intrusive. We asked: What if a 20‑second voice note could flag issues before they escalate? VoiceVitals was born from that question.

What It Does

  • Patients record short voice check-ins that stream into Firebase Storage.
  • Doctor and patient dashboards subscribe to Firestore for live vitals, notes, and playback.
    -ML/DL trained models rate clarity, prosody, and respiratory strain, surfacing a risk badge per recording.

How We Built It

  • Persona-aware React dashboards running dual Firebase configs (doctor vs. patient).
  • Cloud Functions that issue signed upload tokens, validate metadata, and trigger a Python ML microservice.
  • Firestore schema for clinics, doctors, patients, patientAudio, seeded via CLI.
  • Python pipeline (FastAPI + PyTorch) trained on curated speech-health datasets; results sync back to Firestore.

Challenges We Ran Into

  • Deep learning models underperformed on limited data, and the time to learn took up the majority of our weekend. We are also excited to train models for parkinsons, asl, etc...
  • Firebase Hosting throttled our first deploy until we optimized bundle sizes and caching headers.
  • Reinstalling Python/CUDA after I literally cooked my hard drive the other day, and Pip was acting like it didn't exist
  • Clean clinical speech data is scarce, forcing heavy augmentation and bias-aware evaluation.

Accomplishments We’re Proud Of

  • Hit ~70% accuracy distinguishing “healthy” vs. “concerning” recordings.
  • Demo personas (doctor.demo@voicevital.health, patient.demo@voicevital.health) now show real Firestore data.
  • Project is now awaiting research funding to pursue clinical trials/testing, along with more tests to get more data.

What We Learned

  • Persona-specific Firebase clients simplify security reviews compared to role branching in one app.
  • Smaller, well-labeled datasets beat massive synthetic ones when clinicians need explainability.
  • Containerized environments are lifesavers when hardware fails mid-hackathon.

What’s Next

  • Secure funding to run a longitudinal study with partner clinics to improve dementia accuracy.
  • Expand data collection to cover respiratory, speech-motor, and cognitive impairment cohorts.
  • Add disability-specific UX modes (ALS, Parkinson’s, post-stroke) so guidance adapts to each patient.
Share this project:

Updates