Cadence

General Speech Analyser Tab
Reading Practice Tab
Customized Speech Diagnostic Test Tab
Front UI Home
History and Tracking Tab

Inspiration

80 million people worldwide have speech impediments. Speech therapy costs $150+ per session with months-long waitlists. We wanted to build a free, instant, browser-based tool that anyone can use to understand their speech patterns and practice on their own terms — no appointments, no cost, no judgment.

What it does

Cadence analyzes speech recordings to detect four types of disfluency — blocks (silent pauses mid-speech), repetitions (sound/word/phrase), prolongations (stretched sounds), and fillers (um, uh, like). It scores fluency 0-100, visualizes events on an interactive waveform, and provides a detailed breakdown. Beyond analysis, Cadence includes a Learn system that diagnoses your specific speech patterns, assigns personalized courses across 5 difficulty levels, and tracks your progress over time. The History page shows daily and weekly trends in an Apple Screen Time-style view. A Judge Mode transparency panel exposes the entire pipeline — model decisions, latency breakdown, confidence scores, and side-by-side transcript comparisons — so you can see exactly how every detection was made.

How we built it

We built a custom two-tier analysis pipeline from scratch — no LLM wrappers, no GPT calls. Tier 1 (Rules-Based): Audio preprocessing → faster-whisper transcription with word timestamps → energy-based VAD for block detection → Levenshtein distance repetition detection (sound, word, and phrase subtypes) → keyword + prefix filler matching → autocorrelation + spectral flatness prolongation detection → CMUDict syllable-based speaking rate analysis → weighted composite scoring. Tier 2 (Machine Learning): wav2vec2-base embeddings fed into a 6-class RandomForest classifier (97.9% F1 score on synthetic training data). wav2vec2-base-960h CTC decoding for character-level phonetic transcription that preserves stuttering patterns Whisper normalizes away. Google Cloud Speech-to-Text as a third transcription source — cross-referencing all three using the ROVER technique (Recognizer Output Voting Error Reduction). The Learn system uses diagnostic analysis to profile impediment types, then assigns targeted courses (Blocks, Prolongations, Repetitions, Fillers) with exercises that progress from single syllables to 60-second impromptu speech. Users need three consecutive passing scores (≥80) to advance each level. Frontend: Next.js 14 with a "Clinical Glass" design system — glassmorphism cards, Instrument Serif + Geist typography, framer-motion animations, wavesurfer.js waveform visualization, and recharts for progress tracking. Full accessibility support with ARIA labels, keyboard navigation, high contrast mode, and reduced motion respect. Google Cloud Integration: Cloud Speech-to-Text as a dual transcription engine in the analysis pipeline, Cloud Storage for audio persistence, Firestore for production session and progress storage, and Cloud Run for containerized deployment with pre-cached ML models.

Challenges we ran into

--Whisper normalizes stutters. "I I I want" becomes "I want" — the exact patterns we need to detect get erased. We solved this by adding wav2vec2 CTC and Cloud STT as additional transcription sources that preserve raw speech patterns. --False positive blocks. Natural 400-600ms pauses between sentences were flagged as blocks. We tuned the threshold to 600ms and added contextual filters checking for adjacent speech. --Training data. The SEP-28k stuttering dataset requires downloading hours of podcast audio. With limited hackathon time, we pivoted to synthetic training data — generating 1,200 clips with programmatic stuttering modifications, producing 5,905 windowed embeddings for classifier training. --Python 3.9 compatibility. A X | None type union syntax (Python 3.10+) crashed the pipeline on teammate machines. Caught during cross-machine testing and fixed with Optional[X].

Accomplishments that we're proud of

A complete custom DSP + ML pipeline with zero LLM dependencies 6-class disfluency classifier achieving 97.9% weighted F1 Dual ASR cross-referencing that catches patterns either model alone would miss A full learning system with diagnostic assessment, 4 courses, 5 levels, and progression tracking Judge Mode that makes every pipeline decision transparent and auditable

What we learned

Speech disfluency detection is a fundamentally different problem than speech recognition. ASR models are optimized to produce clean text, they actively work against detecting the patterns we care about. Building a reliable detection system required layering multiple approaches (rules + ML + multiple ASR engines) and carefully tuning thresholds against both false positives and false negatives.

What's next for Cadence

Real-time streaming analysis with sub-500ms in-ear feedback so users hear when a block starts while still speaking. A clinician portal where speech-language pathologists monitor patient sessions remotely and adjust course progression, turning Cadence into a between-appointment practice tool that feeds data back to the therapist. Phoneme-level lisp and articulation detection using IPA-trained wav2vec2 models. Personalized classifier fine-tuning on each user's own confirmed disfluency events over time. Clinical validation against SLP-annotated ground truth to pursue insurance reimbursement eligibility. Multilingual support via XLSR-53 for the billions of people who stutter in languages with zero accessible digital tools.