-
-
212 sandboxed trials with no hints: Codex fixed 211. Terra's median is 14.5 s, about the same as chat's head start.
-
The judge site replays recorded rounds and recomputes the benchmark in your browser. No install, no key.
-
Under the hood: EventSub in, Helix out, the Codex SDK under a permission profile, Game Inspector MCP, replay tests, Polly.
-
Live on Twitch, Nov 1: a viewer from another team broke the game and named the bug before Codex fixed it.
-
Boss round: two Codex models race on the same bug while chat bets. On Nov 1, Terra beat Sol in 22.1 s.
-
!break splices a real, pre-tested one-line bug into the game's TypeScript. Chat gets a 15 s head start to guess it.
-
Codex works in a sandbox: tests read-only, no network, no access to the original code. A canary probe checks it.
-
Pit Wall buzzes my phone when chat breaks the game, with veto and hints. iPhone on the live stream, Android in a TEST round.
-
Codex's real SDK steps stream onto the overlay, including calls to my Game Inspector MCP server.
-
Replay tests pass, the fix swaps into the running game, and the autopsy card shows the real diff. Green in 28.47 s.
-
Chat guessed gravity-sign during the head start and won the round. Names are blurred on request.
On most streams, chat just watches. Here a viewer types !break and a real bug lands in the source code of the game the streamer is playing. Chat gets 15 seconds to guess what broke. Then Codex jumps in, finds the bad line and fixes it live, and the game only swaps back once the tests say the fix is real.
Replay a real round in your browser, no install and no key: https://chat-vs-codex-judge.vercel.app Watch it run on Twitch: a viewer beats Codex to the bug (https://www.twitch.tv/steve_sociall/clip/SpoopyConfidentBobaLitty-mrwNi8kcsFCEbvn5) and a boss round where two Codex models race (https://www.twitch.tv/steve_sociall/clip/StrongOpenLasagnaJonCarnage-RfFf1HVjaLdZhpAm).
Codex does all of this in a sandbox where it can't read the original code, can't edit the tests and has no network.
Judge it in 3 minutes
- Watch the film (https://youtu.be/25q8o5AT_1w). 0:00 a real viewer breaks the game and beats Codex. 0:57 the one-line bug. 1:19 how Codex built it, the benchmark and a boss round. 1:57 under the hood, with the exact Twitch and Codex names. 2:13 for streamers.
- Replay Sunday's viewer round in your browser, no install and no key: https://chat-vs-codex-judge.vercel.app/#r=ac8b50c7-1012-4f20-af7e-10dd718b9f1c
- Recompute the benchmark from the committed results: https://chat-vs-codex-judge.vercel.app/#numbers (or run
pnpm bench:summaryin the repo). - Check the build: the main Codex thread is
01a19004-23e3-7883-89a7-f08faee625a5. The commits are at https://chat-vs-codex-judge.vercel.app/#build and the receipt isdocs/BUILD-RECEIPT.jsonin the repo.
What a round looks like (Design: easy to use, fun to watch)
There are three ways to break the game. Type !break in chat, redeem the "Break the game" Channel Points reward, or whisper a bug type so nobody sees it coming.
The server picks one real, pre-tested bug and splices it into the game's TypeScript. The game reloads broken right there on stream. Maybe the controls flip. Maybe an enemy walks straight off a ledge. Chat argues about it with !guess while a symptom replay shows what went wrong.
After 15 seconds Codex comes in with a short bug ticket. The ticket says which replays failed, the first frame where things went off and what viewers saw. It never names the bug. If chat guessed during the head start, the most popular guess rides along as a one-word hint. When it's done, replay tests decide, including one replay Codex never gets to see, and if they pass, the fixed code swaps into the running game without resetting the player.
In a boss round, two Codex models race each other on the same bug. On Nov 1, GPT-6.1 Terra beat GPT-6.1 Sol in 22.1 seconds.
Every fixed round leaves something behind: an autopsy card with the real diff and the time to green, a stream marker at the break and another at the fix, an auto-clip with its link posted in chat, and the result pinned.
Why it's new (Innovation)
Most AI streams put chat and the AI on the same team. Chat asks for something and the AI builds it. I flipped that. Chat causes the problem and Codex has to clean it up, so chat finally has someone to beat. The bugs are real one-line changes to the source, and a fix only counts when a replay proves it.
For streamers and their chat (Potential Impact)
Setup is Node 22, pnpm and the Codex CLI on the streamer's computer, one OBS browser source, and a one-time Twitch sign-in with a device code. The app makes the Channel Points reward for you and gives viewers their points back if a round can't run. I run it on my own regular channel, which isn't Affiliate or Partner, so you don't need either one. Chat gets a real job. The streamer gets a clip and two markers out of every fixed round without doing anything.
Another builder, Blucca, tried the judge site cold and told me two things a first-time viewer needed: the race result right next to the green time, and a way to see the actual glitch. I shipped both the same night (Oct 31), and the site now shows a BROKEN / FIXED replay side by side.
How I built it with Codex (Technological implementation)
Codex built almost all of this in one main thread, 01a19004-23e3-7883-89a7-f08faee625a5. I decided what got built and in what order. Codex wrote the game, the replay tests, the bug pipeline, the benchmark, the overlay, the Twitch layer, the judge site and a second game. The phone app came from a second Codex thread. Every turn got checked against CI before the next one started, and the live parts got tested on stream. At submission that's 20 turns in the main thread, 109 commits on main and 787 tests running in CI on every push.
Codex is also the opponent. Each round starts a fresh run through the Codex SDK. Over 212 sandboxed trials on my pool of 53 bugs, with no hints, Codex fixed 211. The one miss was a dropped network connection, and I kept it in the count. GPT-6.1 Terra at low effort had a median of 14.5 seconds to a green fix, which is about the same as chat's head start, and 91 of its 158 fixes came in under 15 seconds. So it's a real race. In 150 of Terra's 158 fixes and 52 of Sol's 53, Codex put back the exact line that was planted. The rest were different fixes that still passed every replay. Anyone can recompute all of this with pnpm bench:summary, which only reads the committed results, so there's no model call and no key.
The tech
Twitch EventSub and Helix
EventSub brings in chat messages, Channel Points redemptions and whispers the moment they happen: channel.chat.message, channel.channel_points_custom_reward_redemption.add and user.whisper.message. Helix handles everything going back out. It sends chat messages, creates the reward, settles or refunds each redemption, drops stream markers, makes the clip and pins the result, using chat/messages, channel_points/custom_rewards, streams/markers, clips and chat/pins.
OpenAI Codex
The builder. Almost the whole repo came out of one main Codex thread (the phone app came from a second one), and the session ID above lets you check it.
Codex SDK
At runtime the SDK runs the defender. It streams Codex's real steps to the overlay so chat can watch it work, and at the end it hands back a structured diagnosis of what broke.
The Codex sandbox
Codex runs under a custom permission profile. It can read the tests but can't edit them, it can't read the original code, and it has no network. A canary test checks those limits from inside the sandbox.
Game Inspector, my MCP server
During a round Codex can call a small read-only MCP server I built. It replays frames, finds the first frame where the broken game drifts from the good one and compares telemetry. In the Nov 1 viewer round, Codex called first_divergence, diff_telemetry and replay_frames before it touched the code.
Stryker and the bug gates
Stryker suggests candidate bugs, and most of them don't make it. A bug has to compile, change how the game behaves, fail a replay, show up on screen within 600 frames and leave the player alive. 53 made the pool. Bugs can also be generated and checked live when a round starts.
Deterministic replays
The game runs at a fixed 60 Hz with integer physics. Same inputs, same frames, every time, which is what lets a test prove a fix.
PixiJS, Vite and OBS
PixiJS and Vite build the game and the overlay, and the whole thing sits in OBS as one browser source.
Amazon Polly
When it's turned on, Polly reads short calls like "Something broke."
Pit Wall, the streamer's phone app (Expo / React Native)
My phone buzzes the moment chat breaks the game, shows the head-start countdown and Codex's steps, and lets me veto a round or send Codex a one-word hint. The iPhone build also has a Live Activity widget written in Swift that shows the score, the last result and deaths on the lock screen. It pairs with the round runner over the local Wi-Fi with a six-digit code. iPhone: public TestFlight link. Android: APK on the GitHub release.
Vercel judge site
The judge site replays recorded rounds in your browser and recomputes the benchmark from the committed results. No install, no key.
A Codex skill for other games
A hotfix-ready Codex skill gets other small TypeScript games ready for this. I tried it on an original Breakout game and Codex fixed 3 of 3 bugs. Three trials is a small sample, so I'm treating it as a start.
Challenges and what I learned
Going live kept wiping my channel. OBS's Twitch dock had old values saved, so every time I went live it overwrote my title, category and tags, and I now reset them from the API right after. One live round took 1:14 because another Codex build was eating my laptop's CPU at the same time. I don't run builds during live rounds anymore. Whispers need setup too. The viewer needs a verified phone, and the streamer has to accept whispers from them.
The biggest thing I learned is that a fair AI race is way harder to build than a flashy one. Most of the work went into hiding the answer from Codex, proving every fix with replays and giving chat a real head start. That's also the part that makes it fun to watch.
What's next
Getting more live chats onto the raid rounds that are already in the code, where a raid arms a bug for the incoming chat. Shared Chat, so two streamers' chats can break each other's games. And opening the hotfix-ready skill up to indie devs, so they can mark which parts of their own game chat is allowed to break.
Demo transparency
- Who was in chat: on Sunday Nov 1, a viewer from another entrant's team broke the game and named the bug before Codex fixed it (their chat message said their AI agent was typing for their human). Their guess came during the head start, so it went to Codex as a hint. Saturday's rounds were tests I triggered myself, labelled TEST on screen. The boss round was started by me with
!boss. The Android phone shot is a TEST round I ran on an emulator. - The short game calls ("Something broke.") are Amazon Polly.
- The benchmark is repeated trials on one machine and one level. The second game has 3 trials.
- The film adds captions, zooms and music on top of real screen recordings.
Built With
- amazon-polly
- codex-sdk
- eventsub
- expo.io
- github-actions
- node.js
- obs
- openai-codex
- pixi.js
- playwright
- react-native
- stryker
- swift
- twitch
- twitch-helix-api
- typescript
- vercel
- vite
- vitest