Inspiration

Most "Twitch Plays" games suffer from two fundamental bottlenecks:

  1. Input polling chaos: Viewers spam static button presses (up, down, a, b) over high-latency video streams without any continuous physics agency.
  2. Static game rules: Once a stream starts, the game's mechanics are frozen. If viewers or the streamer want to turn the arena into a zero-gravity ricochet chamber or summon a new multi-shield Raid Boss mid-fight, they can't—without stopping the server, rewriting code, and risking a crash.

We built ChaosForge // Anomaly Engine with OpenAI Codex to turn every Twitch chat room into a live-compiled 60 FPS cooperative vector Raid Boss arena where viewers don't just play the game—they trigger safe natural-language mutations to its physical laws and Boss AI mid-stream while an automated safety guardrail guarantees zero downtime.


What it does

ChaosForge // Anomaly Engine runs in two synchronized views—a Split Studio (60 FPS Arena + 4-Card Control Panel) for live direction and a 1920×1080 OBS Clean Overlay (with 1-click Alpha Transparency) for compositing directly over a live Twitch broadcast:

  1. Real Twitch IRCv3 WebSocket + Two-Way OAuth Chat (wss://irc-ws.chat.twitch.tv:443):

    • Connect to any public Twitch channel immediately with zero setup, or authorize via Twitch OAuth (/auth/twitch/start) for two-way live chat messaging directly from the studio.
    • Every viewer who chats in the connected Twitch channel spawns a personalized, color-coded 60 FPS Kinetic Vector Drone orbiting the central Raid Boss.
    • Viewers pilot their drones using real-time kinetic verbs in Twitch chat:
      • !join — Locks a viewer's vector drone into orbital phase.
      • !slingshot — Fires a tangential orbital slingshot burst straight into the Boss core.
      • !tether @user — Forms a cooperative spring-physics harmonic whip between two viewers; crossing the Boss core shears off massive HP.
      • !parry — Emits a phase-reversal wave that reflects Boss projectiles back at the core for 2.4x counter-damage.
      • !overclock & !emp — Boosts drone thrust or shatters rotating Boss shield arcs.
  2. Twitch EventSub Webhook Engine & In-Studio Event Triggers (POST /api/twitch/eventsub):

    • Built to process HMAC-SHA256 verified Twitch EventSub notifications (verified against Twitch-Eventsub-Message-Signature using timing-safe crypto.timingSafeEqual) alongside live in-studio broadcast triggers:
      • channel.raid — Spawns an entire squadron of synchronized Raider Drones into orbit.
      • channel.subscribe — Fires a high-damage Orbital Ion Strike directly onto the Boss core.
      • channel.cheer — Unleashes a radial Bit Shockwave scaled by cheered bits.
      • channel.hype_train.begin — Activates an arena-wide 1.8x Kinetic Overdrive multiplier.
  3. Codex Live Rule Mutator (!patch <natural language>):

    • Streamers or viewers submit natural-language rule prompts—such as !patch Invert gravity into zero-g hyper-elastic ricochet chamber with prism beams and leviathan coil boss—either in Twitch chat or in Card 2.
    • Our hybrid OpenAI / Groq Cloud LLM + Deterministic Semantic AST Compiler compiles natural-language prompts into a validated, bounded ArenaPatchSchema v1 JSON AST that mutates gravity topology (central_attractor, binary_pulsar, magnetic_dipole, zero_g_drift, harmonic_lattice), Boss chassis (hex_citadel, leviathan_coil, chronos_tesseract, gyro_seraph), shield arcs, projectile behaviors, and audio harmonics in real time.
  4. Structural AST Diff Summary & 1-Click Live Rollback (Card 3):

    • Every compiled !patch generates a live Parameter Comparison Diff (Previous Value ➔ New Value) across all physical constants and Boss parameters.
    • Streamers can click Rollback on any historical patch for a one-click instant rollback to previous physics states without interrupting the 60 FPS arena.
  5. Safety Guardrail & Lyapunov Physics Stability Wall (Card 4):

    • Before any compiled AST touches the 60 FPS simulation loop, it must pass automated security, schema, and physics stability invariant checks (validateAndNormalizePatch()).
    • Code Injection Protection: Payloads containing executable or prototype-pollution tokens (eval(), Function(), window, process, __proto__, constructor) are intercepted and blocked before execution.
    • Lyapunov Energy Clamping: Divergent physics mutations (e.g., 9999N gravity or 2.85x super-elastic restitution that would cause infinite kinetic energy explosions or NaN coordinates) are mathematically clamped to stable bounds (gravityStrength <= 420N, restitution <= 1.35, and automatic damping <= 0.982 enforcement whenever restitution > 1.0 to prevent super-elastic kinetic energy divergence).

How we built it

We used OpenAI Codex (gpt-5.4 inside the Codex Desktop engineering environment, Session ID: 01a19c48-01f0-7132-b253-e3b0e83d5bc6) as our core engineering co-designer and verification harness:

  • Zero-Dependency Full-Stack Architecture: Built on Node.js 22 native HTTP + SSE/WebSocket streaming, HTML5 2D Vector Canvas (requestAnimationFrame fixed-timestep physics), and Web Audio API FM synthesis—requiring zero heavy frontend frameworks and verifying that estimated solver cost stays strictly within the < 1.50ms/tick 60 FPS frame budget (FRAME_BUDGET_60FPS).
  • Designing the Guardrail Compiler (src/compiler/schema.mjs & src/compiler/codex-mutator.mjs): Working iteratively with OpenAI Codex, we designed a strict JSON AST validator and semantic compiler that bridges live LLM structured outputs with deterministic fallback synthesis and automated structural diffing (diffPatches()).
  • Continuous Test-Driven Verification (test/chaosforge.test.mjs): Codex continuously ran npm test across our 5 end-to-end verification suites covering safe natural-language !patch compilation, malicious eval() injection blocking, divergent Lyapunov energy clamping, Twitch IRC command parsing, and HMAC-SHA256 EventSub signature verification.

Challenges we ran into

  • Preventing LLM Hallucinations from Crashing a Live 60 FPS Physics Loop: Allowing viewers or an LLM to mutate gravity, restitution, and Boss attack patterns mid-frame can easily introduce NaN velocities or unbounded kinetic energy. We solved this by decoupling natural-language compilation from the physics loop: the LLM never emits raw JavaScript—only a declarative, bounded ArenaPatchSchema AST that passes through our Lyapunov stability wall before hot-swapping.
  • Seamless Two-Way Twitch Chat + Zero-Auth Public IRC: Judges and streamers needed a way to test live Twitch channels immediately without complex OAuth setup, while also supporting two-way chat writing from the studio. We engineered a dual-path Twitch bridge (src/twitch/irc-bridge.mjs & src/twitch/oauth.mjs) that connects anonymously to any public channel via wss://irc-ws.chat.twitch.tv:443 and upgrades to authenticated two-way chat as soon as the user connects via Twitch OAuth.
  • Background Tab Throttling During Live Multi-Window Demos: When streamers switch windows to type in Twitch popout chat, browsers throttle requestAnimationFrame and accumulate backlog frames. We capped delta-time (dt <= 50ms) and throttled secondary DOM list re-renders so switching back from Twitch chat to the 60 FPS arena is instantaneous.

Accomplishments that we're proud of

  • Zero-Downtime Live AST Hot-Swapping: Rewriting the gravitational topology and Boss state machine live on stream with zero frame drops and one-click instant rollback.
  • Transparent Security & Physics Telemetry: Instead of hiding the AI compiler behind a black box, Cards 2, 3, and 4 expose the exact prompt, the compiled AST response, the parameter-by-parameter diff (Previous ➔ New), and the live Guardrail diagnostic JSON report.
  • Broadcast-Ready OBS Alpha Mode: One click strips away the studio chrome and turns the canvas background transparent so streamers can layer ChaosForge directly over their webcam or gameplay in OBS Studio.

What we learned

  • How to strictly bridge non-deterministic LLM natural-language capabilities with hard real-time deterministic game loops through declarative JSON AST boundaries.
  • The power of automated invariant validation: checking Lyapunov energy conservation and string prototype pollution at the schema boundary completely eliminates runtime crashing bugs in live audience environments.

What's next for ChaosForge // Anomaly Engine

  • Twitch Extension Overlay: Packaging the control surface as a native Twitch Video Component Extension so viewers can click to vote on upcoming !patch mutations directly on the video player.
  • Multi-Streamer Federated Boss Raids: Allowing two Twitch channels to merge their orbital arenas during a channel.raid event so both communities battle a shared Mega-Anomaly together.

OpenAI Codex Verification

  • OpenAI Codex Session ID (/status): 01a19c48-01f0-7132-b253-e3b0e83d5bc6

Built With

Share this project:

Updates

Submission history