ChatStage

Every Twitch chatter gets a voice and a face. A browser-only OBS overlay that speaks chat with local neural TTS and phoneme-synced characters. No server, no API key, no TTS fees.

Comment

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for ChatStage

Inspiration

Chat text-to-speech on Twitch is usually robotic (OS voices), paid per character (cloud TTS), or English-first. Japanese and multilingual streams are poorly served, and nothing gives a regular viewer a persistent "self" on stream. I maintain piper-plus, an open-source neural TTS that runs in the browser, so I wanted to see what a stream feels like when chat literally takes the stage.

What it does

  • Reads a channel's Twitch chat (anonymous, read-only, no sign-in) and speaks every message with a neural voice that runs entirely in the streamer's browser / OBS Browser Source.
  • Every chatter gets their own character: a stable voice (8 presets), their Twitch name color, and a mouth that moves phoneme by phoneme using the TTS model's duration predictor. A speech bubble highlights the words as they are spoken.
  • Six spoken languages (Japanese, English, Chinese, Spanish, French, Portuguese), detected per message. Viewers can pick a voice with !voice 1-8, set a language with !lang, or opt out with !quiet.
  • Moderation-aware: bans, timeouts and deleted messages are removed from the queue and never spoken. Mods and the broadcaster can !skip, !pause, !resume, !clear, !ttsmute. Streamers can limit the mic to subs or VIPs/mods.
  • Cleans chat before speaking: Twitch emotes, URLs, emoji, and "wwwwwww"/"8888888" spam.
  • A setup page generates the OBS URL in under a minute, and a scripted demo mode lets anyone try it without a live channel.

How I built it

I built it with OpenAI Codex (CLI) in one thread. I wrote a one-page brief (concept, constraints, features in priority order, quality bar). Codex read the piper-plus package source to use the real API (PCM samples, sample rate, phoneme timing, lengthScale), wrote the vanilla-JS modules (IRC parser, text cleaner, language detection, voice presets, command/moderation controller, cancellable speech queue with prefetch, SVG stage and viseme lip-sync), wrote 37 node:test tests, ran the site in Chrome to check real synthesis and phoneme sync, and tested the anonymous Twitch connection. When live mode failed on the deployed site, Codex reproduced it in Chrome against a live channel, found that a browser timer was called with the wrong receiver ("Illegal invocation") before the chat handlers were attached, and separated audio readiness from chat startup so a retry reconnects both. In a follow-up turn in the same thread, Codex switched every character voice to the public-domain CSS10 model to respect a voice corpus's terms, and made the static site work under a GitHub Pages subpath.

No build step: plain ES modules and an importmap, deployed to GitHub Pages. TTS is piper-plus (MIT) with ONNX Runtime Web + WebAssembly; pitch variety comes from synthesizing longer and compensating with playbackRate, so each voice keeps its pace.

Challenges

  • Lip-sync had to follow the actual audio clock, not timers, so pause/skip/clear never desync the mouth.
  • Twitch emote positions are counted in Unicode code points, not UTF-16 units; an emoji earlier in the message shifts everything.
  • A message can be deleted while its audio is still being synthesized; the queue cancels it before it ever plays.
  • Voice licensing: one candidate voice's terms require consultation before voicing other characters, so ChatStage uses only the public-domain CSS10 voice for characters.

What's next

More voice models (Korean and Swedish are detected but not spoken yet), custom character skins for channel points, and an EventSub mode for channel-point redemptions.

Built With

Share this project:

Updates

Submission history