VoxTrace

VoxTrace live dashboard with evidence-backed delivery scores and temporal comparison.
Public GitHub repository with source, licensed dataset preview, tests, and technical report.

Inspiration

Public-speaking feedback is often subjective: a speaker is told to “slow down” or “sound more confident” without being shown where the problem occurred or what changed in the signal. VoxTrace was built to make delivery feedback auditable. It compares two recordings of the same script—an ideal baseline and a target delivery—and turns measurable acoustic differences into timestamped practice guidance.

What it does

VoxTrace accepts 16-bit PCM WAV recordings and extracts bounded acoustic windows. It measures energy, silence, pitch variation, mean F0, spectral centroid, spectral rolloff, spectral flatness, zero-crossing rate, and compact cepstral coefficients. When genuine word timestamps are available, it also measures speech rate and word-boundary pauses.

The comparison engine maps target windows to the ideal same-script recording, derives an explicit reference profile, and reports the observed value, baseline value, and delta. A deterministic scoring layer identifies rushed delivery, excessive pauses, monotone delivery, low energy, and low clarity only when the required measurement exists. Missing measurements remain visibly missing.

How we built it

The backend uses Python, FastAPI, NumPy, and an optional WhisperX phoneme-alignment worker. Uploads are size- and duration-limited, processed through temporary files, and deleted immediately. A React and TypeScript dashboard presents the overall rubric score, data completeness, baseline-vs-target energy timeline, timestamped findings, causal evidence, and prioritized practice actions.

For the checked-in dataset preview, we selected three LibriSpeech utterances distributed under CC BY 4.0 and generated five deterministic contrastive variants per reference. This produces 18 WAV recordings across three pseudonymous speakers and scripts. Generated temporal labels stay visibly pending until human review. Evaluation is recording-scoped and uses a speaker-held-out split.

Challenges

The main challenge was preserving scientific honesty under hackathon time pressure. It is easy to display impressive-looking confidence values or stretch transcript words evenly across an audio file. VoxTrace does neither. Forced alignment has explicit provenance, the resource-heavy model is isolated behind a guarded optional endpoint, and every unsupported metric is omitted.

Another challenge was comparing recordings with different durations. The deterministic baseline comparison maps windows by normalized same-script time, while the forced-alignment path provides word-level timing for speech-rate and pause analysis.

Accomplishments

Working ideal-vs-target audio comparison API and dashboard.
Exact timestamped causal findings with no invented metrics.
FFT-derived spectral descriptors, compact cepstral coefficients, F0, energy, silence, pitch variation, and word-timing contracts.
Optional WhisperX phoneme forced alignment with strict server-side gating.
An 18-recording licensed stress preview with source IDs, licence provenance, and deterministic transformations.
A clearly labelled speaker-held-out synthetic stress test: macro F1 1.000 and mean matched IoU 1.000. This verifies pipeline integrity and is not a natural-speech accuracy claim.
26 passing backend tests and a verified production frontend build.
Public live demo, public source repository, five-page technical report, and transparent AI-assistance disclosure.

What we learned

Temporal grounding is not the same as attaching a timestamp to generic feedback. The timestamp, feature value, reference value, threshold, and explanation must share the same provenance. Data quality and review status matter more than simply collecting many recordings, so the pipeline keeps generated labels pending until a human reviews them.

What's next

A future natural-speech benchmark will add consented same-script recordings and independently reviewed temporal labels before any real-world accuracy claim. Longer-term work includes multilingual alignment profiles, microphone-condition normalization, richer synchronized transcript highlighting, and user-controlled encrypted persistence.

Responsible use

VoxTrace is an educational coaching prototype, not a medical, psychological, employment, or competition-judging system. Delivery conventions vary across cultures, languages, speaking styles, microphones, and rooms. Raw speech can contain biometric information and must not be published without permission and an appropriate licence.

AI assistance disclosure

ByteAsk / ChatGPT and OpenAI Codex assisted with planning, architecture discussion, code and test support, audit work, documentation, and production of the narrated demo. Human contributors remain responsible for reviewing the implementation, licensing, measurements, and every submission claim.