Run It Back

Chat types !moment and Codex turns the stream's funniest fail into a brand-new mini-game, live on the overlay. Chat plays it, remixes it, and it lands back in the streamer's game.

Comment

Inspiration

The best moments on a stream last ten seconds. The streamer misses the easiest jump, chat floods KEKW, and it's gone. Clips save the video, but nobody gets to play that moment. We wanted chat to grab it and run it back, as a game, on stream, while everyone is still laughing.

What it does

  • A viewer types !moment <what happened>. The bot answers right away and the overlay shows a live FORGING card.
  • Run It Back grabs the frame from a few seconds ago (and, for our demo game Blob Jump, the real replay trace of the run) and hands it to Codex. Codex writes a complete custom mini-game for that exact fail: its own mechanic, rules, labels and hidden answers, as code against a small sandboxed runtime.
  • While Codex works, its real progress streams onto the card and chat predicts "Will chat beat this moment? W/L".
  • Before anything airs, the game is sandboxed and play-tested headless, and it has to be winnable and losable.
  • Chat plays three rounds by voting. Voters show up as avatars, MVPs get credited, and the bot posts the result in chat.
  • The result lands back in the streamer's game: a trophy or a gravestone appears at the exact spot of the fail.
  • For 90 seconds chat can !remix <change> ("the floor is lava"). Codex edits its own game, the card shows the real code diff, and v2 ships.
  • It isn't tied to our game: through OBS WebSocket any game works. The demo shows SuperTuxKart.
  • Every moment stays in the channel's arcade, where anyone can replay it.

How we built it

  • Twitch: EventSub WebSocket (channel.chat.message, stream.online) to read chat, Helix Send Chat Message with an app token + user:bot / channel:bot so replies carry the Chat Bot badge, automatic token refresh. The overlay is a browser source in OBS.
  • Codex at runtime: every !moment and !remix is one codex exec run with the frame image, the replay trace and the cartridge JSON schema (--output-schema), in a minimal environment with no Twitch secrets. Codex's reasoning summaries are read from the JSONL event stream and shown on the overlay. Nothing on that card is faked.
  • Safety gate before anything goes on stream: Ajv schema check, an acorn AST allow-list (no globals, DOM, network or eval; bounded loops only), a sandboxed iframe with CSP and a heartbeat kill switch, then a headless Playwright play-test that must reach both a win and a loss and checks real motion, text contrast, overlapping labels and blank screens. A failure goes back to Codex for one repair; a verified preset game is the last fallback, so something verified always ships.
  • Any game: OBS WebSocket screenshots of the game source replace the replay trace.
  • Building it with Codex: the project was built in Codex, prompt by prompt. After each iteration we ran the full loop end to end, screenshotted the overlay, and wrote the next prompt from what was actually broken. Codex wrote the code and the tests.
  • Node 22 + TypeScript, flat-file storage, Kenney CC0 sprites, Web Audio sounds.

Challenges we ran into

  • Running model-written code live in front of an audience safely: sandbox + AST guard + headless play-test before it airs.
  • Making each game look like the moment (real level backdrop, real hero sprite, the real fail spot) and not like a generic template. We caught Codex copying prompt examples into most games and added a diversity check.
  • Dead air: 30–45 seconds of generation had to become part of the show, so the live Codex feed and W/L predictions fill it.
  • Real-stream issues: 120 Hz displays producing too many frames, OBS 32 image MIME types, occluded browser windows that stop rendering.
  • A streamer-vs-chat duel mode exists (DUEL_MODE=1) but is off by default: fresh duels still fail our fairness gate too often, and we'd rather ship a verified game than an unfair one.

Accomplishments that we're proud of

  • A viewer's sentence becomes a verified, playable, moment-specific game on a real Twitch stream in about 30–45 seconds.
  • Chat directs Codex to rewrite its own game live, with the real diff on screen.
  • The same loop works on a game we didn't write (SuperTuxKart) through OBS.

What we learned

Verification is what makes runtime code generation broadcast-safe. And showing the model's real work is more entertaining than hiding it behind a spinner.

What's next for Run It Back

Fair streamer-vs-chat duels, a Twitch Extension so viewers play on their own screens, and per-channel seasons and leaderboards.

SuperTuxKart is GPLv3 / CC-BY-SA. Music in the video: Juhani Junkala (CC0).

Built With

Share this project:

Updates

Submission history