-
-
Structural AST Diff Summary: Inspect live parameter changes (Previous ➔ New) across physics & Boss AI with 1-click rollback.
-
Real Twitch IRCv3 connection: Viewers typing !join and !slingshot in #offerastudio spawn drones and attack the Raid Boss live.
-
ChaosForge Split Studio: 60 FPS vector arena driven by live Twitch chat alongside the 4-card Codex AST compiler & control panel.
-
Guardrail Safety Wall: Intercepts forbidden eval() code injections and enforces Lyapunov kinetic energy bounds with zero downtime.
-
OBS Alpha Mode: 1-click transparent browser source compositing the 60 FPS Raid Boss directly over live Twitch stream broadcasts.
Inspiration
Most "Twitch Plays" games suffer from two fundamental bottlenecks:
- Input polling chaos: Viewers spam static button presses (
up,down,a,b) over high-latency video streams without any continuous physics agency. - Static game rules: Once a stream starts, the game's mechanics are frozen. If viewers or the streamer want to turn the arena into a zero-gravity ricochet chamber or summon a new multi-shield Raid Boss mid-fight, they can't—without stopping the server, rewriting code, and risking a crash.
We built ChaosForge // Anomaly Engine with OpenAI Codex to turn every Twitch chat room into a live-compiled 60 FPS cooperative vector Raid Boss arena where viewers don't just play the game—they trigger safe natural-language mutations to its physical laws and Boss AI mid-stream while an automated safety guardrail guarantees zero downtime.
What it does
ChaosForge // Anomaly Engine runs in two synchronized views—a Split Studio (60 FPS Arena + 4-Card Control Panel) for live direction and a 1920×1080 OBS Clean Overlay (with 1-click Alpha Transparency) for compositing directly over a live Twitch broadcast:
Real Twitch IRCv3 WebSocket + Two-Way OAuth Chat (
wss://irc-ws.chat.twitch.tv:443):- Connect to any public Twitch channel immediately with zero setup, or authorize via Twitch OAuth (
/auth/twitch/start) for two-way live chat messaging directly from the studio. - Every viewer who chats in the connected Twitch channel spawns a personalized, color-coded 60 FPS Kinetic Vector Drone orbiting the central Raid Boss.
- Viewers pilot their drones using real-time kinetic verbs in Twitch chat:
!join— Locks a viewer's vector drone into orbital phase.!slingshot— Fires a tangential orbital slingshot burst straight into the Boss core.!tether @user— Forms a cooperative spring-physics harmonic whip between two viewers; crossing the Boss core shears off massive HP.!parry— Emits a phase-reversal wave that reflects Boss projectiles back at the core for2.4xcounter-damage.!overclock&!emp— Boosts drone thrust or shatters rotating Boss shield arcs.
- Connect to any public Twitch channel immediately with zero setup, or authorize via Twitch OAuth (
Twitch EventSub Webhook Engine & In-Studio Event Triggers (
POST /api/twitch/eventsub):- Built to process HMAC-SHA256 verified Twitch EventSub notifications (verified against
Twitch-Eventsub-Message-Signatureusing timing-safecrypto.timingSafeEqual) alongside live in-studio broadcast triggers:channel.raid— Spawns an entire squadron of synchronized Raider Drones into orbit.channel.subscribe— Fires a high-damage Orbital Ion Strike directly onto the Boss core.channel.cheer— Unleashes a radial Bit Shockwave scaled by cheered bits.channel.hype_train.begin— Activates an arena-wide 1.8x Kinetic Overdrive multiplier.
- Built to process HMAC-SHA256 verified Twitch EventSub notifications (verified against
Codex Live Rule Mutator (
!patch <natural language>):- Streamers or viewers submit natural-language rule prompts—such as
!patch Invert gravity into zero-g hyper-elastic ricochet chamber with prism beams and leviathan coil boss—either in Twitch chat or in Card 2. - Our hybrid OpenAI / Groq Cloud LLM + Deterministic Semantic AST Compiler compiles natural-language prompts into a validated, bounded
ArenaPatchSchema v1JSON AST that mutates gravity topology (central_attractor,binary_pulsar,magnetic_dipole,zero_g_drift,harmonic_lattice), Boss chassis (hex_citadel,leviathan_coil,chronos_tesseract,gyro_seraph), shield arcs, projectile behaviors, and audio harmonics in real time.
- Streamers or viewers submit natural-language rule prompts—such as
Structural AST Diff Summary & 1-Click Live Rollback (Card 3):
- Every compiled
!patchgenerates a live Parameter Comparison Diff (Previous Value ➔ New Value) across all physical constants and Boss parameters. - Streamers can click Rollback on any historical patch for a one-click instant rollback to previous physics states without interrupting the 60 FPS arena.
- Every compiled
Safety Guardrail & Lyapunov Physics Stability Wall (Card 4):
- Before any compiled AST touches the 60 FPS simulation loop, it must pass automated security, schema, and physics stability invariant checks (
validateAndNormalizePatch()). - Code Injection Protection: Payloads containing executable or prototype-pollution tokens (
eval(),Function(),window,process,__proto__,constructor) are intercepted and blocked before execution. - Lyapunov Energy Clamping: Divergent physics mutations (e.g.,
9999Ngravity or2.85xsuper-elastic restitution that would cause infinite kinetic energy explosions orNaNcoordinates) are mathematically clamped to stable bounds (gravityStrength <= 420N,restitution <= 1.35, and automaticdamping <= 0.982enforcement wheneverrestitution > 1.0to prevent super-elastic kinetic energy divergence).
- Before any compiled AST touches the 60 FPS simulation loop, it must pass automated security, schema, and physics stability invariant checks (
How we built it
We used OpenAI Codex (gpt-5.4 inside the Codex Desktop engineering environment, Session ID: 01a19c48-01f0-7132-b253-e3b0e83d5bc6) as our core engineering co-designer and verification harness:
- Zero-Dependency Full-Stack Architecture: Built on Node.js 22 native HTTP + SSE/WebSocket streaming, HTML5 2D Vector Canvas (
requestAnimationFramefixed-timestep physics), and Web Audio API FM synthesis—requiring zero heavy frontend frameworks and verifying that estimated solver cost stays strictly within the< 1.50ms/tick60 FPS frame budget (FRAME_BUDGET_60FPS). - Designing the Guardrail Compiler (
src/compiler/schema.mjs&src/compiler/codex-mutator.mjs): Working iteratively with OpenAI Codex, we designed a strict JSON AST validator and semantic compiler that bridges live LLM structured outputs with deterministic fallback synthesis and automated structural diffing (diffPatches()). - Continuous Test-Driven Verification (
test/chaosforge.test.mjs): Codex continuously rannpm testacross our 5 end-to-end verification suites covering safe natural-language!patchcompilation, maliciouseval()injection blocking, divergent Lyapunov energy clamping, Twitch IRC command parsing, and HMAC-SHA256 EventSub signature verification.
Challenges we ran into
- Preventing LLM Hallucinations from Crashing a Live 60 FPS Physics Loop: Allowing viewers or an LLM to mutate gravity, restitution, and Boss attack patterns mid-frame can easily introduce
NaNvelocities or unbounded kinetic energy. We solved this by decoupling natural-language compilation from the physics loop: the LLM never emits raw JavaScript—only a declarative, boundedArenaPatchSchemaAST that passes through our Lyapunov stability wall before hot-swapping. - Seamless Two-Way Twitch Chat + Zero-Auth Public IRC: Judges and streamers needed a way to test live Twitch channels immediately without complex OAuth setup, while also supporting two-way chat writing from the studio. We engineered a dual-path Twitch bridge (
src/twitch/irc-bridge.mjs&src/twitch/oauth.mjs) that connects anonymously to any public channel viawss://irc-ws.chat.twitch.tv:443and upgrades to authenticated two-way chat as soon as the user connects via Twitch OAuth. - Background Tab Throttling During Live Multi-Window Demos: When streamers switch windows to type in Twitch popout chat, browsers throttle
requestAnimationFrameand accumulate backlog frames. We capped delta-time (dt <= 50ms) and throttled secondary DOM list re-renders so switching back from Twitch chat to the 60 FPS arena is instantaneous.
Accomplishments that we're proud of
- Zero-Downtime Live AST Hot-Swapping: Rewriting the gravitational topology and Boss state machine live on stream with zero frame drops and one-click instant rollback.
- Transparent Security & Physics Telemetry: Instead of hiding the AI compiler behind a black box, Cards 2, 3, and 4 expose the exact prompt, the compiled AST response, the parameter-by-parameter diff (
Previous ➔ New), and the live Guardrail diagnostic JSON report. - Broadcast-Ready OBS Alpha Mode: One click strips away the studio chrome and turns the canvas background transparent so streamers can layer ChaosForge directly over their webcam or gameplay in OBS Studio.
What we learned
- How to strictly bridge non-deterministic LLM natural-language capabilities with hard real-time deterministic game loops through declarative JSON AST boundaries.
- The power of automated invariant validation: checking Lyapunov energy conservation and string prototype pollution at the schema boundary completely eliminates runtime crashing bugs in live audience environments.
What's next for ChaosForge // Anomaly Engine
- Twitch Extension Overlay: Packaging the control surface as a native Twitch Video Component Extension so viewers can click to vote on upcoming
!patchmutations directly on the video player. - Multi-Streamer Federated Boss Raids: Allowing two Twitch channels to merge their orbital arenas during a
channel.raidevent so both communities battle a shared Mega-Anomaly together.
OpenAI Codex Verification
- OpenAI Codex Session ID (
/status):01a19c48-01f0-7132-b253-e3b0e83d5bc6
Built With
- html5
- javascript
- node.js
- obs-studio
- openai
- openai-codex
- twitch
- twitch-eventsub
- twitch-irc
- web-audio-api
- websockets