Chat City

Twitch chat codes a city live on stream. A viewer types !build and a description, and Codex writes them a real 3D building while chat watches the crane work.

Comment

Inspiration

Chat is the best part of a stream, and it scrolls away. Someone makes the joke of the night and ten minutes later it's gone. I wanted viewers to leave something that stays: a building with their name on it, in a city that grows every stream.

What it does

A viewer types !build a cozy bubble tea shop with a giant glowing cup on the roof. The request lands on the streamer's desk, and nothing happens until the streamer clicks Approve.

Then Codex goes to work. The empty plot on the overlay gets scaffolding and a crane, and a ticker shows what Codex is doing right now: writing the building, running its checks, fixing a problem. When it's done, the building rises out of the ground with the viewer's name above it. The bubble tea shop in my demo took 87.9 seconds and passed on the first try.

Twitch events change the city too:

  • Raid: a new district opens at the edge of town with a gate named after the raider. Codex builds it a landmark, and for 10 minutes new chatters build there.
  • New sub: that viewer's building gets a light beam and gold trim. No building yet? They get a free build that skips the line.
  • Hype train: a real train circles the city, one car per level, until the hype train ends.
  • Channel points: a reward buys a build that skips the queue.

The city is saved between streams. It also exports to a public page where anyone can spin the city around, click a building, and see who made it, what they asked for, and the exact code Codex wrote.

The streamer stays in charge. They can approve, skip or demolish from the desk, !demolish is mods-only, and auto-approve is off by default.

How I used Codex

Codex does two jobs here.

It built the app. I built Chat City in one main Codex thread (session ID in the form), breaking the work into smaller tasks as I went: the Twitch client, the validator, the queue, the overlay and the export.

It's the engine on stream. Every approved !build starts a new Codex SDK thread in its own folder. That folder has a short contract with the rules for a building, an example, and check.js, which is the same validator my server uses. So Codex can test its own building before it says it's done. Each thread runs with network and web search off, in a workspace-write sandbox, on low reasoning effort, because speed matters live.

Codex's progress events go straight to the overlay as ticker lines, so chat watches it work instead of staring at a spinner. If my validator rejects the result, the error goes back to the same thread and the overlay says "Codex is fixing it". It gets one retry. If it fails twice, the viewer gets their slot back.

Every job's time and token usage is logged. I'm on the free Codex plan, so there are limits: 2 builds at once and 25 per stream.

How I built it

Plain JavaScript on Node 25, with no framework and no bundler. Four dependencies: @openai/codex-sdk, three, ws and dotenv.

  • Twitch EventSub over WebSocket for chat, raids, subs, channel points and hype trains. Chat replies go through the Helix API ("@name your bubble tea shop is going up on plot 14").
  • Three.js for the city. Buildings use only basic shapes and colours, no models or textures. The whole city shares one night palette, so random requests still look like one place.
  • A transparent OBS browser source for the overlay, plus a full-screen city cam.
  • 38 tests with node:test covering the validator, chat command parsing, the queue and the Twitch client.

Challenges I ran into

Running code that chat asked for. Both the viewer's text and the code Codex writes are untrusted. Chat text is cut to 200 characters and handed to Codex as a labelled description, never as instructions. The code then goes through two checks. First, a banned-word scan (no import, fetch, eval, timers, window and so on) and a 30 KB size limit. Second, a separate Node process locked down with Node's permission model: it can only read, it has no network and no child processes, and it gets 3 seconds. That process runs the building and measures it: at most 6,000 triangles, 80 meshes, one 1×1 plot, 3 units tall. I found out late that Node 22 couldn't block network access the way I needed, so the validator requires Node 25. To be straight about it: this is a denylist plus a permission sandbox, not a full JavaScript sandbox. That's why streamer approval is on by default.

Waiting is boring. A build takes a minute or more (58–88 seconds in my measured runs). So the wait had to be part of the show: scaffold, crane, live ticker, then the rise with dust and a name tag.

Testing without a big channel. I used the Twitch CLI mock for raids, subs and hype trains, but it can't send chat messages. So I built a /dev page that fakes chat through the same code path.

Accomplishments that I'm proud of

Codex isn't hidden behind the feature. You watch it work on stream. Every building keeps its author, the original request and its code, so anyone can check what Codex actually wrote.

What I learned

The safety rules and the waiting experience took more work than the 3D did. Giving Codex the exact check script my server runs was the most useful single decision: it can catch its own mistakes before the building ever reaches the stream.

What's next for Chat City

  • Run building code in a real worker or container, so it's safe to run without moderation on bigger channels.
  • An !upgrade command, so viewers can add to their building on later streams.
  • A full live run on a real channel with real viewers.

A note on the demo video

The chat and Twitch events in the video are simulated on my machine and labelled that way on screen. They go through the same code path as live EventSub. The Codex build is real.

Testing instructions

Needs Node 25+ and a Codex login (your ChatGPT account). No Twitch account or API key is needed to try it locally.

npm ci
cp .env.example .env
codex login
npm start
  1. Open http://localhost:3000/dev and send a !build message.
  2. Open http://localhost:3000/dashboard.html and click "Approve build". This runs a real Codex job (about 1 to 1.5 minutes).
  3. Watch it at http://localhost:3000/overlay.html?full=1

The /dev page also has buttons for a raid, a sub and a hype train. To connect a real channel, follow "Twitch setup" in the README.

Built With

Share this project:

Updates

Submission history