SpeakQuest

Take score + slate: five rubric-based dimension scores, flags and points lost. Same take, same score, every time.
The transcript is the chart: width = pace, weight = loudness, height = pitch, gaps = pauses. Reference shown above your take.
Flagged regions hatched on the waveform with exact timestamps. Each note shows the measured numbers (0.55x ref pace) and a drill.
Script library: 8 scripts from warm-up to hard, each listing the flaws it tests and its reference length.
Recording booth: script card, reference playback and a live input level meter.
Recording live. The tally light follows the actual input level.
Take recorded. Play it back, re-take with R, or send it for analysis.
Live pipeline progress: normalise, transcribe, analyse, explain, save.
Take history with scores and flags. A clean read scores 100; a dragged read with a filler scores 86.

Inspiration

Anyone who has judged or competed in elocution or debate knows delivery feedback is vague. "Slow down", "more energy". Two judges hear the same speech and score it differently. We wanted feedback that points at the exact second something went wrong and says what was measured.

What it does

You give SpeakQuest a reference reading and a participant's reading of the same script. It lines the two up word by word and flags where the participant drifts: rushing, dragging, long pauses, flat pitch, trailing off, skipped words and fillers. Each flag has a start and end time and the numbers behind it, for example the pause was 970 ms where the reference paused 140 ms.

Scores for pace, pauses, pitch, volume and accuracy come from a fixed rubric file, so the same recording always gets the same score. An LLM (Amazon Bedrock) only writes the coaching text, and a validator throws its output away if it mentions any number that wasn't actually measured.

On the results page the transcript itself shows the delivery. Rushed words are squeezed, slow words stretched, loud words heavier, pauses are real gaps, and flaws are underlined.

Dataset

We couldn't find recordings with per-flaw time labels, so we built our own. 8 scripts, 56 clips, about 10 minutes: a reference reading, two clean readings by other voices, and four flawed versions per script. We generated speech with Piper TTS and read the model's internal phoneme durations, which gives us the exact start and end of every word. Then we injected flaws at word boundaries with controlled strength (speeding up or slowing down a span, inserting pauses or "um", flattening pitch, fading volume, cutting words). Because we injected them, the labels are exact.

How we built it

ffmpeg cleans up the audio, Vosk gives word timings (Amazon Transcribe in the cloud version), and a Needleman-Wunsch alignment matches spoken words to the script. Praat via parselmouth measures pitch, loudness and pauses. Pitch is measured in semitones from each speaker's own median and loudness relative to their own average, so a deep voice isn't flagged just for being deep.

The backend is written in Terraform for AWS: API Gateway, Lambda, Step Functions, S3, DynamoDB, Transcribe and Bedrock. The frontend is React + TypeScript on Vercel.

Results

We tuned thresholds on 5 scripts and tested once on 3 scripts we hadn't touched. On those held-out scripts, precision was 0.67, recall 0.78 and mean timestamp IoU 0.93, with a median word-boundary error of 24 ms. The weak spot is false alarms on clean readings, 3.51 per minute against our target of under 1, and rushed speech is the hardest flaw to catch.

Challenges we ran into

The AWS account we were given blocks Lambda, S3, DynamoDB, Step Functions, Transcribe and API Gateway at the organization level, so we couldn't deploy the backend. It's all written and passes terraform plan, it's tested against local AWS emulators, and Bedrock coaching works live. The whole system runs end to end locally, and the Vercel site is a labelled demo.

Speech recognizers also like to "clean up" ums and repeated words, which are exactly what we need to catch.