TakeTwo — Compare Speech. Understand the Difference.

Inspiration

Two people can read the exact same sentence and still sound completely different. Even the same person can deliver a passage differently each time, changing their pace, pauses, and emphasis.

I wanted to build something that could make those differences easier to examine.

Most speech tools focus on transcription or give you an overall score. I was more interested in what happens between the words: where timing changes, where energy shifts, and whether those differences can be traced back to an actual moment in the recording.

That's where TakeTwo started.

What it does

TakeTwo is a local speech-analysis workspace that compares a reference recording with another recording of the same passage.

Instead of producing a vague score, it focuses on measurable evidence.

You can use TakeTwo to:

• Compare two recordings side by side.
• Inspect waveforms and acoustic measurements.
• Examine provisional word-level timing differences.
• Find unusual gaps and changes in signal energy.
• Replay specific sections from either recording.
• Review automatic observations alongside their supporting evidence.
• Export a readable analysis summary or a detailed technical report.

The application separates automatic observations from verified conclusions. If the available evidence isn't sufficient to support a reliable score, TakeTwo makes that limitation clear rather than inventing a result.

Everything runs locally, without requiring a cloud AI API.

How I built it

I used Python and FastAPI for the backend, with React, TypeScript, and Vite for the interface. SQLite handles local data and analysis records.

The audio-processing pipeline uses NumPy, SciPy, and librosa to extract acoustic information, including signal energy, recording activity, and timing-related measurements.

For word alignment, I integrated Montreal Forced Aligner (MFA), which estimates where the supplied transcript's words occur in each recording.

I then built a comparison layer that examines corresponding word intervals, groups related acoustic observations, and identifies sections worth listening to.

One important part was making every observation traceable. Users can inspect the measurements, timestamps, and evidence behind a finding instead of simply trusting a label.

I also built a separate human-review workflow and export system to keep automatic analysis distinct from independently reviewed evidence.

Challenges I ran into

The hardest part wasn't displaying waveforms. It was deciding what the application could actually conclude from them.

For example, an automatic word alignment might show a long gap between two words. That doesn't necessarily mean the speaker paused. The alignment could be incorrect, the transcript might not match, or there could be other sounds in the recording.

I had to design the analysis around those uncertainties.

Pitch extraction was another challenge. Some real recordings produced insufficient reliable pitch measurements, so I kept those conclusions unavailable instead of lowering the requirements just to display a result.

There were also practical engineering problems: getting audio playback to use the correct timestamps, preserving analysis results across restarts, handling processing failures, and generating readable reports without losing the technical evidence.

The project went through several iterations because the early versions generated plenty of numbers but didn't do enough to explain which ones were useful.

What I learned

TakeTwo taught me that extracting information and making that information useful are two different problems.

A system can produce dozens of technically correct measurements and still leave the user wondering what to do with them.

I learned a lot about audio processing, forced alignment, temporal comparisons, backend reliability, and building interfaces around complex data.

More importantly, I learned how important it is to distinguish between what a system measures, what it suspects, and what it can actually verify.

That distinction ended up shaping the entire project.

What I'm proud of

I'm proud that TakeTwo has grown into a working application rather than remaining a processing script or a static prototype.

It can analyze genuine recordings, identify inspectable differences, connect findings to the corresponding audio, and produce reports that preserve the underlying evidence.

I also put significant effort into automated testing, reproducibility, and keeping the application usable on ordinary CPU-based hardware.

There are still limitations, especially around independent human validation and reliable pitch analysis, but those limitations are visible rather than hidden.

What's next

The next step is to test TakeTwo with more recordings, additional speakers, and independently reviewed annotations.

I also want to improve the reliability of pitch analysis, finish manual accessibility testing, and evaluate how consistently its highlighted observations help people identify meaningful differences.

Eventually, I'd like TakeTwo to become a practical, open-source tool for examining speech delivery without turning every difference into an unsupported judgment.

For now, the goal is simple: make speech comparisons easier to understand, one recording at a time.

Built With

• acoustic
• aligner
• alignment
• analysis
• audio
• fastapi
• html5
• javascript
• librosa
• montreal
• numpy
• playwright
• processing
• pytest
• python
• react
• ruff
• scipy
• speech
• sqlite
• typescript
• uplot
• visualization
• vite