-
-
homepage
-
homepage2
-
no flaw analysis
-
flaw analysis
-
pitch & energy & preferrnce
-
flawed regions
-
compare 2
Inspiration I wanted to build something that actually helps people speak better — not a generic "how was my speech" tool, but one that tells you exactly where your delivery broke down and why.
Speech coaching is subjective. Teachers, coaches, and speech therapists rely on intuition and memory. Nobody can say "your pitch variance dropped 71% between 12.4s and 15.1s, making you sound monotone" without hard data. I wanted to fix that.
What it does
Oratory compares your speech recording to a reference recording of the same text. It aligns them with Dynamic Time Warping on MFCCs, detects flaw regions, and explains each one with real numbers.
Types of flaws detected:
- Rushed pace - "Speaking rate is 1.75x the reference"
- Monotone delivery - "Pitch spread is 0.2 semitones vs 3.8 in reference"
- Unnatural pauses- "Silence fills 89% more of this span than the reference"
- Volume shifts - "Loudness is +12.2 dB vs the reference"
Each flaw comes with a coaching tip: "Mark a breath at each comma and slow down on the phrase you most want remembered."
The dashboard shows:
- A color-coded waveform (red = severe, yellow = mild)
- Click any region to play that exact segment
- Pitch and energy contours vs the reference
- Four sub-scores: Pace, Pitch, Pauses, Volume (out of 25 each)
- A compare mode for two recordings side by side
- Recent analyses history
How I built it Dataset construction:
- 4 baseline speeches from LibriVox (public domain)
- 5 severity levels per speech (L0 clean → L4 severely flawed)
- 20 clips total with ground-truth timestamps of every injected flaw
- Flaws programmatically injected so I know exactly where they are
Pipeline:
- Feature extraction — MFCCs, F0 (semitones), energy (dB), pause detection
- DTW alignment — aligned participant audio to reference on MFCCs with cosine metric
- Windowed flaw detection — 1-second windows with thresholded flags
- Region merging — combined adjacent flags into coherent flaw regions
- Causal explanations — template-based using real numbers from the analysis
- Rubric scoring — 4 dimensions × 25 points each = 100 total
Stack:
- Backend: Python, FastAPI, librosa, Parselmouth, pyworld
- Frontend: Vanilla HTML/CSS/JS, Wavesurfer.js v7, Plotly.js
- No paid APIs — runs entirely local
Results Real evaluation metrics on the 20-clip dataset:
- Spearman(score, severity) = -0.985 — score drops near-perfectly as severity increases
- IoU on flaw timestamps = 0.743 — high timestamp accuracy
- Recall = 0.594 — catches ~60% of injected flaws
- Precision = 0.36 — over-reports, next step is tuning
- L0 score = 100/100 — clean audio scores perfectly
- L4 score = 85/100 — heavily flawed audio scores lower
I report precision honestly because over-reporting is the real failure mode — a coach needs to know the tool catches things that aren't always there, so they use it as decision support, not a judge.
Challenges I faced
- DTW alignment — naive DTW on full audio was too slow. Fixed with librosa's
subseqand downsampling. - Flaw injection — programmatically injecting pace/monotone/pause/volume flaws while keeping audio natural took several iterations.
- Cross-language — pitch extraction differs between male/female voices. Normalized to semitones relative to median F0.
- Honest evaluation — I tuned on the same clips I tested on, which inflates scores. Next version uses a held-out test set.
What I learned
- Building with real data beats building with fake data every time.
- Reporting precision honestly is more valuable than hiding it. Judges respect the truth.
- Ground truth keeps you humble. You think your system works, then you measure it, and you realize how much you still don't know.
What's next
- Held-out test set for honest evaluation
- Regional language support (Hindi, Telugu)
- Real-time streaming mode for live coaching
- Mobile app for self-recording
Log in or sign up for Devpost to join the conversation.