SpeechLab - Temporal Speech-Flaw Grounding

Pitch contour and acoustic feature visualization for comparing speech delivery.
Evidence-based explanation of detected speech deviations, including confidence scores and measured values.
Interactive loudness and flaw-interval explorer for locating deviations in the candidate recording.
Loudness timeline highlighting detected flaw intervals against the speaker’s baseline.
Interactive spectrogram heatmap for exploring acoustic patterns in speech.
Overview dashboard showing speech-flaw metrics, delivery profiles, and evidence verification.

Inspiration

Speech delivery is more than the words we say. Pauses, speaking rate, pitch variation, and filler words can change how a message is delivered. We wanted to build a tool that makes these changes visible and explainable using audio evidence rather than subjective judgments.

What it does

Track C Audio Studio compares a candidate recording with a speaker's baseline audio to identify measurable speech-delivery deviations. It highlights detected intervals, provides timestamps and confidence scores, and lets users inspect audio segments and visualizations.

How we built it

We built an interactive Python and Streamlit dashboard with audio-processing, forced-alignment, feature extraction, detection rules, and visualization components. The interface combines baseline comparisons, acoustic profiles, loudness plots, pitch and spectrogram views, and evidence cards explaining the measurements behind each flagged interval.

Challenges we faced

One major challenge was connecting a detected deviation to a specific audio interval and presenting evidence that users could inspect. We also focused on comparing recordings against the same speaker's baseline and avoiding unsupported conclusions about a speaker's emotions or personality.

What we learned

We learned the importance of transparent evaluation, speaker-specific baselines, and explanations grounded in measurable acoustic features. A detection is more useful when users can inspect the evidence instead of receiving only a score.

What's next

We would like to expand the dataset, evaluate across more speakers and recording conditions, improve detector calibration, and validate performance on unseen recordings.