Oscine: Contrastive Speech Analytics & Flaw Grounding

Automated contrastive speech engine that temporally bounds oratorical flaws down to the millisecond, delivering causal mathematical coaching against elite orator baselines.

Comment

Inspiration

Speech evaluation has always suffered from two critical failures: subjective variance and feedback ambiguity. Speakers walk away from judging panels with vague critiques like "slow down" or "be more engaging" — zero temporal precision on where delivery broke down and no objective evidence to act on. Existing automated tools assign arbitrary scores without linking them to the acoustic timeline. We built Oscine to eliminate this guesswork — transforming subjective coaching into millisecond-accurate, mathematically grounded speech diagnostics benchmarked against elite rhetorical exemplars.

What it does

Oscine is an automated contrastive speech evaluation engine that temporally bounds oratorical delivery flaws and generates causal coaching rationales. It: Temporally localizes flaws — errant pauses, rushed pacing, and monotone intonation are pinpointed with exact start and end timestamps, not vague summaries. Benchmarks contrastively — evaluates spoken delivery against 6 culturally diverse orator baselines (US, UK, Indian, and Nigerian English) rather than a rigid robotic tempo. Generates causal explanations — translates raw acoustic signal shifts into human-readable coaching points backed by measurable pitch, volume, and pause duration deltas. Scores on a 10.0 composite rubric — deterministic multi-dimensional scoring across Pacing, Intonation, Dynamic Range, and Continuity. Visualizes interactively — a synchronized multi-track dashboard links waveform scrubbing (WaveSurfer.js) with pitch and energy contour curves (Apache ECharts) for intuitive timeline inspection.

How I built it

Oscine is an automated contrastive speech evaluation engine that temporally bounds oratorical delivery flaws and generates causal coaching rationales. It: Temporally localizes flaws — errant pauses, rushed pacing, and monotone intonation are pinpointed with exact start and end timestamps, not vague summaries. Benchmarks contrastively — evaluates spoken delivery against 6 culturally diverse orator baselines (US, UK, Indian, and Nigerian English) rather than a rigid robotic tempo. Generates causal explanations — translates raw acoustic signal shifts into human-readable coaching points backed by measurable pitch, volume, and pause duration deltas. Scores on a 10.0 composite rubric — deterministic multi-dimensional scoring across Pacing, Intonation, Dynamic Range, and Continuity. Visualizes interactively — a synchronized multi-track dashboard links waveform scrubbing (WaveSurfer.js) with pitch and energy contour curves (Apache ECharts) for intuitive timeline inspection.

Challenges I ran into

Acoustic splice clicks — naive audio slicing during flaw injection produced audible high-frequency artifacts. Resolved by implementing phase-aligned zero-crossing boundary crossfades.

Temporal drift with Dynamic Time Warping — standard DTW distorted intentional rhetorical pauses, misclassifying deliberate emphasis as disfluency. Replaced with a two-tier approach: forced word alignment for semantic anchors, followed by local prosodic deviation tracking.

Chart-audio synchronization — high-frequency audio playhead updates caused visual stuttering in the ECharts renderer. Resolved by throttling crosshair updates via requestAnimationFrame and pre-downsampling acoustic envelopes

Accomplishments

60-clip calibrated benchmark — the first dataset pairing elite rhetorical baselines with systematically engineered disfluencies at calibrated severity levels and verified millisecond ground-truth boundaries. Fully deterministic grounding — every detected flaw is backed by a verifiable acoustic delta. No black-box predictions. Sub-second evaluation — pre-cached reference feature files deliver analysis responses well under one second after user audio upload. Synchronized multi-track dashboard — pitch, volume, and waveform tracks all respond to a single playhead in real time with smooth 60fps scrubbing.

What I learned

How to quantify acoustic prosody variance to mathematically distinguish natural expressive inflection from monotone delivery collapse. High-throughput time-series processing using vectorized NumPy operations across raw audio frames — critical for keeping analysis latency low. Designing explainable AI feedback loops where every system output maps to a specific, verifiable physical measurement.

What's next for Oscine: Contrastive Speech Analytics & Flaw Grounding

Real-time WebRTC streaming — live browser microphone input for in-the-moment rehearsal feedback without file uploads.

Volume instability detection — extending the flaw taxonomy to flag sudden dynamic collapses and projection inconsistencies mid-utterance.

Multi-genre corpus expansion — adding courtroom argumentation, academic lectures, and startup pitching baselines to broaden evaluation coverage.

Technical Report: Read Full Project Report (PDF)

Built With

Share this project:

Updates

Submission history