Inspiration
My grandmother was diagnosed with dementia at the end of the previous year. She died a couple of months ago.
There was a special connection between us that was not there between me and any other family member. Whenever we visited each other, my siblings would go out to play while she and I would sit and have long conversations. Once, during Christmas, we made gingerbread houses so poorly that we laughed at how bad they were.
Her dementia was slowly creeping up on her since the beginning of 2025. At Easter, she called me Ashish, which is my father's name, instead of calling me Arav, and no matter how many times I corrected her, she was convinced she was right. This happened throughout the year until the diagnosis was finally made. During one of her last days, when I went to visit her in the hospital, she did not even recognize me.
Reflecting on that, the signs were present in her speech months earlier than anyone could see them: long pauses, repetitions of sentences, the same stories multiple times during the day. It takes specialized technology to detect a 12% change in someone's speech rate. People simply do not notice it until the point where the signal becomes undeniable because the illness got several years of a head start.
What it does
Sona is a voice-based early warning system that operates solely in your web browser. You record three brief, ad-lib takes — Sona simply prompts you to say something like "Tell me about your day" — and from these recordings it extracts 16 different acoustic and linguistic properties: speaking rate, pauses, filler words, repeated words, lexical diversity, pitch, and vocal effort.
It analyzes the numbers in four different ways: in relation to your personal historical data (for a Voice Stability score), in relation to a small reference corpus of typical speech, in relation to a logistic regression model built on dementia speech research, and finally in relation to your recorded data history over time. If more than three dementia-specific indicators start moving in the same direction simultaneously, a sustained trend flag goes off. It cannot be triggered by a single anomaly.
Absolutely nothing is sent anywhere but your device. There is no account, no server, no database — everything exists client-side in localStorage and is one-click deletable.
How we built it
Next.js, React, and TypeScript, single-page, without a backend. The audio pipeline is implemented manually: using MediaRecorder to capture input, and then using the Web Audio API to compute RMS framing, adaptive silence thresholding, pausing, and autocorrelation-based pitch tracking, all performed client-side. Variability in pitch and energy is represented in terms of coefficients of variation to enable comparability between different voices and microphone configurations.
The feature extractor is implemented relative to a small structural interface, rather than directly using AudioBuffer, making it compatible with NodeJS. This made it possible to implement the comparison between reference speech by running the same pipeline on 200 random LibriSpeech audio clips and computing percentile breakpoints, instead of having two implementations of the same math.
The dementia classifier is a logistic regression classifier trained on labeled clinical speech data, with the class distribution calibrated to match the direction and magnitude of change reported in Fraser et al. 2016 and Ding et al. 2024.
Challenges we ran into
The Chrome Speech Recognition API quietly turns off mid-record even with the continuous option on, which is why most of a 60-90 second recording was being returned as an empty transcript. We solved this problem by restarting a new recognizer each time the previous one closed until the user stopped recording.
Pitch detection is an O(n²)-ish search problem, and trying to run it at full microphone sample rate led to a freeze of several tens of seconds for a 90-second clip. Reducing the sampling rate to ~11 kHz - since speech pitch is well under 1 kHz - reduced this time to just hundreds of milliseconds.
Perhaps the hardest decision was what metrics to compare between reference speech and recorded speech. LibriSpeech is a short, scripted, read-aloud narration; Sona recordings are long and spontaneous. A read sentence cannot have a pause in it, so comparing the pause count metric to it would give a confident but meaningless result. Of 16 metrics we tested against the dataset, only 3 made it through.
Accomplishments that we're proud of
We coded the entire signal processing pipeline from scratch, without using any audio library or machine learning toolkit, and it's fast enough to work live in a browser tab. Every number displayed by Sona originates from audio; there is no fabrication.
This trend flag, which requires more than four recordings and more than three agreeing signals, is therefore silent most of the time, by design. This is precisely what we wished had happened with my grandmother in early 2025 instead of at Easter.
What we learned
Extraction of pause count is done in 30 lines of code. However, understanding what the pause count meant – in comparison to what, in comparison to whom, on what scale – accounted for most of the 40 hours. The danger with a number which does not have a good reference class is that it is equally convincing as any other number.
What we learned mostly is that the problem is tractable. It is a real thing that can be measured using nothing more than a laptop microphone and some mathematics.
What's next for Sona
Access to real speech samples of people diagnosed with dementia (DementiaBank/Pitt), retraining to the real base rate rather than a theoretical 50/50. Longitudinal tracking to support the hypothesis of a trend over several months, not just a single attempt. Passive monitoring – included in a routine family phone call, not something that you need to launch manually. Collaborative caregiver perspective, because it's often the people close to someone who first observe symptoms of dementia in them, and not vice versa. And in the future – the same system for Parkinson's, aphasia and other conditions that affect the voice.
Sona will never make any diagnoses. It's meant to say one sentence, two years ahead of what a family would normally notice: it may be time to visit a doctor. For my grandmother, that sentence came too late. This is why Sona exists.
Log in or sign up for Devpost to join the conversation.