StutterClear

Real-time and recorded stutter removal in your own voice. NVIDIA Nemotron on Nebius checks every cut, prepares your words, and helps when you're stuck.

Comment

Inspiration

About 1% of adults stutter, around 70 million people. Video calls, interviews and voice messages are where it costs the most: people switch words, avoid calls, or skip speaking up. Existing tools either retrain speech over months or replace your voice with a synthetic one. We wanted the listener to hear you, just without the repeats, and to keep every word you meant.

What it does

  • Live: speak into your mic; StutterClear plays your own voice about half a second later with repeated words and broken-off starts removed, to headphones or into Zoom/Meet via a virtual cable. A Sensitivity bar sets how hard it cuts, even while you talk. A Fluency Mirror shows what you said, what was removed, and what the listener hears; NVIDIA FastConformer captions show what the listener hears, live. A call can't play what you haven't said yet, so there each removed stutter becomes a short pause; played back as a recording, that time is closed up too.
  • Voice notes: record or upload a message; get it back cleaned, cutting only between words.
  • Meaning guard: before a recording is cleaned, Nemotron reviews every proposed cut and keeps repeats that were meant ("that that is true", "had had", "very very"). Anyone can try it on a typed sentence under "How StutterClear works".
  • Word rescue: when you get stuck on a word, Nemotron suggests the few words you most likely mean, from what you've said and your context card. You choose.
  • Say it: if you want, NVIDIA Magpie TTS says the word you chose, always labelled as a synthetic voice.
  • Context agent: say where you'll speak; Nemotron (with Tavily search) prepares the names and terms you'll need.
  • Interview practice: Nemotron plays an interview partner so you can rehearse speaking out loud.
  • Insights: three short, supportive notes after a session. No diagnosis, no cure claims.

How we built it

  • Detection: a WavLM-Large teacher with a BiGRU head, 5 stutter classes, trained on Kaggle GPUs with SEP-28k / FluencyBank, synthetic stutters spliced into LibriSpeech, and silver labels we derived from CrisperWhisper verbatim transcripts of real stuttered speech.
  • Live: the teacher runs on sliding windows (4 s past + 200 ms lookahead, a decision every 160 ms; 60 ms per window on a laptop RTX 4050). A splicer delays your voice, skips what was marked, never plays audio before it has been judged, rebuilds the delay in natural pauses, and fills gaps with your own room tone.
  • Recordings: the teacher finds stutters; the verbatim transcript places every cut on a word boundary; Nemotron's meaning guard reviews each cut.
  • NVIDIA Nemotron 3 Super on Nebius Token Factory in five roles: meaning guard, word rescue, context agent, interview practice, insights. Text only, never audio. NVIDIA FastConformer (NeMo streaming, via sherpa-onnx) for live captions. NVIDIA Magpie TTS for "Say it".
  • Hosting on free resources: no free always-on GPU host was available to us (we tried Hugging Face Spaces), so the GPU comes to the app when someone needs it. The app runs on Google Cloud Run (CPU, Mumbai); voice notes, Dictate, video and the big live model wake a private Kaggle notebook (2x T4) that connects out to the app over a signed WebSocket and sleeps again when idle. A wake takes 2-4 minutes, shown step by step on the page; after that a 20 s voice note comes back in about 10 s. A static front door on GitHub Pages is the link that never shows an error.
  • App: FastAPI + WebSockets; React UI with a WebGL "liquid" background, the ElevenLabs orb, live waveform mirror, A/B playback.

Challenges we ran into

  • Jerky edits. Our first cuts sounded jerky. Measuring every join showed why: the model's frame edges are ~100 ms off (a syllable), so 43% of cuts sliced into a real word. Placing cuts on word boundaries from the transcript removed every sliced word.
  • Live vs. lookahead. A small causal student was too weak (it lost 8% of words); running the big teacher on short sliding windows kept nearly all of its offline quality.
  • A stutter slipping through. An end-to-end test showed the GPU model's warm-up let undecided audio play; the splicer now never plays audio the detector has not judged.

Accomplishments that we're proud of

On 7 speakers the models never trained on, through the real live pipeline:

stutters removed words lost
Live, rule-based baseline 7.6% 1.25%
Live, teacher on GPU, Gentle (keeps fillers) 33.1% 3.10%
Live, teacher on GPU, Strong (fillers too) 65.2% 5.68%
Live, teacher on GPU, Sensitivity bar at 100: Gentle / Strong 59.5% / 77.3% 9.98% / 12.96%
Recordings, word-aligned (teacher v2): Natural / Thorough (fillers, repeated phrases) 57.2% / 90.9% 0 words sliced

The cut thresholds were chosen on these same speakers, so these numbers may be slightly optimistic. On the standard speaker-exclusive SEP-28k-E test set (thresholds tuned on validation speakers only), teacher v2 reaches macro F1 0.618 (AUC 0.98 for word repetitions, 0.96 for fillers).

What we learned

Listening beats metrics: the version that scored best on paper sounded worse, and the reason was measurable once we looked at the joins. And a stutter is not a repeated word: only a language model can tell "I I I want" from "that that is true".

What's next for StutterClear

User testing with people who stutter (and their speech therapists), a smaller live model for laptops without a GPU, Zoom/Meet one-click setup, and more languages.

Data and credits

Real recordings from UCLASS, the University College London Archive of Stuttered Speech (Howell, Davis & Bartrip, 2009); data collection was supported by the Wellcome Trust. Used for research and teaching; the recordings are never redistributed. Training data: SEP-28k / FluencyBank and LibriSpeech; silver labels from CrisperWhisper. StutterClear is an assistive tool, not a treatment.

Built With

Share this project:

Updates

Submission history