CodexSwitch

CodexSwitch turns Twitch chat into a chat-driven tabletop broadcast layer. Viewers type actions; a dungeon master (OpenAI when a key is set, otherwise a local resolver) returns narration and a d20 resolution; OBS browser-sources a transparent 1920×1080 overlay with three buses — chat, dm, or both. Spam like 666 stays danmaku; verbs and longer lines become story beats. The sample adventure is The Lantern Below. Mock chat ships by default so judges can run a full demo with npm start and no Twitch account.

Inspiration

Twitch chat is already a second screen for every stream. Chinese live platforms treat that second screen as a first-class visual — flying comments (danmaku) that feel like part of the show. On Twitch, chat usually stays in a side panel while tabletop and “Twitch Plays” streams still need a DM, dice, and a clear on-air layer. CodexSwitch borrows the danmaku energy and wires it to an AI DM so the audience can act in the adventure without leaving chat — and so the streamer gets a clean OBS overlay instead of reading the panel aloud.

What it does

• Ingests Twitch IRC chat (optional) or scripted mock viewers (default).
• Classifies each line: commands (!act, !layer, !density, !status), short spam → danmaku, action verbs / long text → story beats.
• Resolves beats with OpenAI JSON when OPENAI_API_KEY is set; falls back to a local d20 DM so the demo never hard-depends on the API.
• Pushes state over WebSocket + SSE to a transparent /overlay (1920×1080) for OBS Browser Source, plus an operator switchboard at /.
• Lets anyone in chat switch layers live (chat / dm / both) and change danmaku density.

How we built it / How we used Codex

A single Node process: HTTP for the switchboard and overlay, WebSocket /ws and SSE /api/events for live frames, optional Twitch IRC, and a small adventure clock. Codex scaffolded the server layout, overlay/switchboard UI, chat→beat classifier, local DM + OpenAI fallback, Twitch IRC path, and the npm test suite. Codex Session ID: 01a19b79-bfb7-7d00-98b7-150d9ec1c964.

Challenges

Keeping spam as danmaku while promoting real actions; readable transparent OBS overlay; mock-first vs live Twitch behind one router; stream-key/phone gates delayed live push — recorded demo uses the same overlay path as mock chat.

Accomplishments

One npm start demo; optional OpenAI with local fallback; real channel peterzhou1990 on http://140.82.62.54:3000; automated live-server tests.

What we learned

Hard split between noise and intent; design overlay as a stage; keep a single Codex build thread with mock-first loop.

What's next

Unlock stream key / go-live; EventSub hooks; multi-adventure packs.

How to run

git clone https://github.com/happy-star/aidm.git
cd aidm && npm install && npm start

Switchboard http://localhost:3000 · Overlay http://localhost:3000/overlay · Health /api/health · Live deploy http://140.82.62.54:3000/overlay · Twitch https://www.twitch.tv/peterzhou1990

Built With

• html/css/js
• node.js
• obs-browser-source
• openai-api
• server-sent-events
• twitch-irc
• websocket