Speech Flaw Analyzer

Inspiration Generic presentation feedback like "you spoke too fast somewhere" is completely useless. It doesn't tell you exactly where you messed up, or by how much. We were frustrated by critiques based on "vibes," so we wanted to build a tool that compares a reading with a reference and points to the exact seconds where delivery drifts—backing up the feedback with hard, measurable numbers.

What it does Speech Flaw Analyzer is a Streamlit dashboard that takes your speech (either live-recorded or uploaded) and compares it against a reference reading. It specifically hunts down three main deviations: rushed pacing, awkward long pauses, and weird volume drops.

Instead of a vague letter grade, it gives you a visual timeline with red bands highlighting exactly where you slipped up. It tells you the exact start and end times, gives a concrete reason (e.g., "word duration 0.67x of this speaker's usual pace"), and lets you play back those specific chunks so you can hear the flaw yourself.

How we built it We knew we wanted an explainable system, so we skipped training a massive black-box AI classifier and built a strict rule-based comparison engine.

The Baseline: We used two readings of the Gettysburg Address and wrote a script to inject controlled flaws (like 1.5x speed rushes or massive volume drops) to create our dataset.

The Tech: We used faster-whisper for word-level timestamps and matched words to the baseline using difflib.

The Math: We normalized pitch (in semitones) and loudness relative to each speaker's own median, ensuring different voices could be compared fairly. Then, we set fixed thresholds—like flagging a pause if it's 0.8 seconds longer than the baseline, or catching a 9 dB drop over a 6-word window.

Challenges we ran into Building this as a two-person team sharing a single laptop threw plenty of hurdles at us.

Python 3.14 Compatibility: faster-whisper's audio decoder flat-out failed on our Python version. We had to pivot and use librosa to load the audio array before passing it to Whisper.

Noisy Timestamps: Short words and silences produced massive fake volume drops and artificially short durations. We had to implement a 6-word averaging window for loudness and demand a run of at least 3 fast words before officially flagging a rushed pace.

Honest Benchmarking: Over-flagging on clean speech was a huge headache, forcing us to constantly re-evaluate and tune our rules so we didn't penalize normal speaking cadence.

Accomplishments that we're proud of We built an analyzer where every single flag is 100% explainable by construction. We hit massive accuracy on the flaws we injected ourselves, scoring an IoU of 0.924 for rushed pace, 0.88 for pauses, and 0.943 for volume drops. We're also proud of our honest evaluation: when testing two completely clean readings against each other, the detector only falsely flagged 3 regions.

What we learned We learned that detecting flaws we intentionally inject ourselves is the easy part. The true test of an analyzer is comparing two clean readings by different people—that's where you see how badly the detector over-flags. We learned the hard way how crucial it is to validate every claim against a fresh run and to report our false-positive numbers openly.

What's next for Speech Flaw Analyzer Right now, our thresholds are hand-picked and our test files were relatively short 15-second clips. Next up, we are going to test injected flaws across full-length readings, add more diverse speakers, and properly tune our thresholds on a held-out dataset to drastically reduce false alarms on clean, free-flowing speech.