-
-
main
-
main
-
main
-
main
Inspiration
Speech feedback often tells someone to “improve their timing” or “use more energy,” without explaining where to improve.
SpeakTrace grew from a simple idea: connect feedback to a specific moment that the speaker can see, hear, and understand. We wanted to make speech comparison transparent, accessible, and possible on a normal laptop without paid APIs.
What it does
SpeakTrace compares a reference recording with another reading of the same English text.
It aligns words to their locations in both recordings, measures delivery differences, and presents findings through an interactive dashboard.
Users can:
- Compare pacing, pauses, and relative energy.
- Select a flagged interval and replay the corresponding audio.
- Click transcript words to navigate both recordings.
- Inspect waveforms and energy, pitch, and spectrum charts.
- Adjust scoring weights for the next analysis.
- Export measurements as JSON or open a printable report.
Scores describe similarity to the chosen reference under explicit rules. They are not universal judgments of speaking ability.
How we built it
The application uses a Python FastAPI backend and an HTML, CSS, and JavaScript frontend.
A pretrained wav2vec2 model supplies acoustic information, and CTC forced alignment locates the supplied words in each recording. NumPy and SciPy support timing, energy, pitch, and spectral measurements. Fixed comparison rules produce the findings and scores.
The system checks transcript agreement and alignment reliability. Uncertain sections may be excluded from assessment, with coverage and warnings shown to the user.
Processing runs locally, with CPU support and optional NVIDIA GPU acceleration. No new speech model was trained.
Our custom dataset
We sourced four short public-domain speech excerpts from two archive recordings and created forty comparison versions: four unchanged controls and thirty-six generated edits.
The edits introduce mild, moderate, and severe changes to pauses, pacing, or energy. Their locations are recorded separately for evaluation and are not provided to the detector.
Source recordings stay separated between development and test groups. The dataset includes transcripts, transformation labels, provenance, and file checksums.
Challenges and what we learned
Reliable word alignment is essential: incorrect word boundaries can produce misleading feedback.
We also discovered that artificially speeding up speech can introduce recognition artifacts. This means a generated pacing example may change more than pacing alone.
Our final development evaluation detected all six pause edits in the test group, one of six energy edits, and none of the six pacing edits. These results identify pause detection as the strongest current feature and expose the work still needed.
This evaluation uses a small synthetic dataset. It is not independent human validation, and the final rerun is not an untouched validation study.
Accomplishments
We built a working local pipeline from audio upload to word alignment, acoustic measurements, timestamped findings, playback, and report export.
Verification passed 104 Python tests, 12 JavaScript tests, and 13 browser checks. The project includes reproducible setup instructions, dataset documentation, evaluation results, and a six-page technical report.
What's next
Improve pacing and energy detection, evaluate consenting human speakers, broaden accents and recording conditions, and compare findings with independent human annotations.
AI assistance
OpenAI Codex and ChatGPT assisted with research, architecture, implementation, testing, design, and documentation.
Log in or sign up for Devpost to join the conversation.