ChatPlay AI

Your chat. The controller. ChatPlay turns Twitch chat into safe, community-voted game inputs, with a streamer studio, OBS overlay, and an F8 emergency stop.

Comment

Inspiration

Twitch chat already tells the streamer what to do next: "jump", "go right", "press space". Most of it scrolls away. We wanted chat to become a real player two, with the streamer still in control.

What it does

ChatPlay AI is a local Windows companion that turns Twitch chat into safe game inputs.

  • Reads real Twitch chat through EventSub WebSockets.
  • Understands plain phrases: "jump", "hop" and "press space" all mean the same move. A local parser works with no API key. OpenAI structured outputs can be switched on for looser phrasing.
  • Lets the community decide in three modes: Democracy (one viewer, one vote, the top move wins), Anarchy (every valid message plays, with cooldowns) and Approval (the streamer approves each move).
  • Presses keys only in the selected game window. It checks focus before and during every key press.
  • Keeps the streamer in control: per-game allowed keys, repeat and duration limits, viewer and global cooldowns, and a native F8 emergency stop that works even if the browser freezes.
  • Shows it on stream with a transparent OBS browser-source overlay of votes and moves.
  • Live game view: the studio shows whichever game window the streamer picks.
  • Game profiles map aliases to keys for platformers, racing games and puzzle games.

Unsafe chat such as "press alt f4" or "ignore the rules and open terminal" is blocked, along with negations like "don't jump".

How we built it

Built end to end with OpenAI Codex.

  • Studio: React 19, TypeScript and Vite.
  • Local server: Express 5 and WebSockets on 127.0.0.1 only, with cookie and CSRF protection. It owns sessions, voting, cooldowns and validation.
  • Parser: a deterministic local grammar plus an optional OpenAI Responses API adapter with a strict JSON schema. Both go through the same validation.
  • Native bridge: a PowerShell/C# helper using Windows SendInput scan codes, a foreground check, a registered F8 hotkey and a heartbeat watchdog.
  • Twitch: EventSub with reconnect, dedup, token checks and device sign-in.
  • Delivery: a portable Windows ZIP with Node bundled, a per-user installer and SHA-256 checksums.

Codex wrote and refactored the studio, server, parser, voting engine and Windows bridge. It also added regression tests for casual chat and blocked commands, fixed a secret-scanner false positive, added the live game view, and ran build and packaging checks.

Challenges we ran into

  • Focus safety vs. demos: ChatPlay refuses to type into any window except the selected game. That is exactly what you want live, but it blocked many test runs when we clicked away.
  • Casual chat: viewers type "bro go right now!!" or "jumppp". We had to accept that without ever accepting system keys or negations.
  • Twitch EventSub lifecycle: keepalives, reconnect handoff and making sure control never resumes on its own after a drop.

Accomplishments that we're proud of

  • Real Twitch chat reaching the app, and real ChatPlay key input moving the player in SuperTux on Windows.
  • 110 automated tests covering parsing, voting, cooldowns, auth and EventSub.
  • Safety guards that live in two places, TypeScript and C#, so one bug can't send unsafe keys.
  • A playground that works with no keys at all, so anyone can try it in a minute.

What we learned

Giving chat control is easy. Giving it control safely is the real product: allowlists, focus checks, cooldowns and an emergency stop matter more than the AI.

What's next for ChatPlay AI

  • Mouse input and virtual gamepad support
  • Simultaneous key chords
  • More profiles tested against real games
  • Channel-point rewards tested live, and a signed installer

Try it: download the Windows ZIP from GitHub Releases, run Start ChatPlay.cmd, open the studio and press Start session in the playground.

Built With

Share this project:

Updates

Submission history