QuestForge

QuestForge turns Twitch streams into live RPGs using OpenAI Codex: chat votes in real time to steer an AI story with persistent memory, dynamic D20 rolls, consequence cascading, and an OBS overlay.

Comment

Inspiration

Live streaming on Twitch has transformed passive viewing into a global community hangout, but interactive stream gaming hasn't evolved beyond simple trivia polls or chaotic button-spam games. Tabletop RPGs (like D&D and Baldur's Gate) are beloved for their deep storytelling, character consequences, and unexpected dice rolls—yet running a live RPG for an audience of hundreds or thousands of viewers is nearly impossible for a single streamer to manage.

We asked ourselves: What if an AI Dungeon Master, powered by OpenAI Codex, could weave a dynamic, infinite fantasy narrative in real time, where the Twitch audience collectively controls the hero’s fate?

That vision became QuestForge—an interactive stream-based RPG platform where chat votes decide narrative branches, dice checks determine life-or-death trials, and past decisions permanently ripple through future chapters.

What it does

QuestForge is an AI-powered interactive RPG broadcast studio and overlay engine:

  1. AI Chronicler with Persistent Memory: Unlike stateless chatbots that suffer from AI amnesia, QuestForge uses an AI narrative memory engine. It remembers past choices, tracks recurring NPCs, maintains consequence flags, and weaves callbacks into future acts.
  2. Real-Time Live Audience Decrees: Viewers vote on the hero's next move through Twitch chat commands (!vote <choice>) or on-screen web buttons. Votes update dynamically with animated probability progress bars.
  3. Interactive D20 Fate Die Checks: Risky actions trigger Difficulty Class (DC) skill checks. An animated physics-style D20 die rolls live on stream, factoring in the character's courage modifiers to trigger Critical Successes or Catastrophic Blunders.
  4. Twitch Chaos Engine: Channel events (raids, subscriptions, follows) spawn live in-game Chaos Events—summoning rogue bosses, dropping ancient relics, or triggering sudden weather shifts.
  5. MDI Multi-Window Desktop Interface: A customizable Multiple Document Interface (MDI) allows streamers and players to drag, minimize, maximize, and tile modules (Story Chronicler, Cartographic Radar Map, Hero Dossier, Twitch Stream Chat, Fate Shapers Leaderboard).
  6. Transparent OBS Esports Overlay: A dedicated /overlay route designed as a zero-opacity Browser Source for OBS and Streamlabs, superimposing live voting bars, health gauges, and narrative beats directly over the stream.
  7. Procedural Sound FX Engine: Uses the native Web Audio API to synthesize crystal pings, dramatic gongs, dice rolls, and victory fanfares without external audio file latency.

How we built it

QuestForge was architected from the ground up using modern web standards and AI:

  • OpenAI Codex: Used to architect the backend prompt orchestration, design the persistent story memory schema, and structure narrative output into strictly validated JSON schemas.
  • Next.js 14 (App Router) & TypeScript: Provides a robust, type-safe full-stack framework with zero compile-time errors.
  • Supabase Realtime & PostgreSQL: Stores session states, player stats, votes, and event logs, broadcasting instant state changes to all connected clients via WebSockets.
  • TMI.js: Interfaces directly with Twitch IRC, parsing chat commands (!vote, !stats, !inventory, !quest, !leaderboard, !chaos, !help) and relaying viewer actions into the game loop.
  • Framer Motion & Tailwind CSS: Powers the cyber-tactical RPG HUD—combining Midnight Charcoal (#17191c), Cool Steel Grey (#929da7), Vibrant Coral Orange (#ff4b32), Warm Cream Beige (#e8d2b5), and Muted Sage Grey (#a5a79b).
  • Web Audio API: Procedurally generates sound effects in real-time in the user's browser.

Challenges we ran into

  1. Eliminating AI Amnesia: Standard LLM prompts lose track of earlier details over multi-round games. We engineered a rolling context builder that injects summarized scene history, active consequence flags, NPC directories, and audience alignment scores into each generation step without exceeding token budgets.
  2. Real-Time Multi-Viewer Concurrency: Handling concurrent votes without race conditions or database bottlenecks required tuning Supabase Realtime subscriptions and aggregating vote tallies with optimistic client-side updates.
  3. MDI Floating Window Drag Physics: Creating a fluid, draggable multi-window desktop interface inside a web browser that supports z-index focus stacking, minimize/maximize animations, and responsive boundaries required deep integration with Framer Motion drag constraints.

Accomplishments that we're proud of

  • Flawless Production Build: All 12 dynamic and static Next.js routes compile cleanly with zero TypeScript warnings.
  • Zero Asset Latency: By synthesizing audio procedurally and rendering SVG cartography, the application runs with virtually zero external asset dependencies.
  • Built-in Crowd Simulator: Designed a realistic simulated chat and voting stream so judges and streamers can experience the thrill of a 500-person live audience even when testing offline.
  • OBS-Ready Utility: Streamers can add /overlay to their OBS studio and start playing within 30 seconds.

What we learned

  • How to structure LLM outputs reliably using OpenAI Codex and TypeScript interfaces to guarantee predictable state transitions in game loops.
  • Advanced state synchronization techniques between Twitch IRC sockets, PostgreSQL real-time tables, and React state.
  • How to design a multi-tiered color system that blends cyber-tactical gaming aesthetics with antique fantasy parchment warmth.

What's next for QuestForge

  • Twitch Bits & Channel Points Integration: Allowing viewers to spend Channel Points to revive fallen heroes or summon companion beasts.
  • Multiplayer Co-op Raids: Allowing two streamers' audiences to battle in the same shared dungeon concurrently.
  • Voice Synthesis & AI Voice Acting: Integrating real-time AI speech generation for the Chronicler and NPC encounters.
  • Twitch Interactive Extension: Deploying QuestForge as an official on-screen Twitch panel extension.

Built With

Share this project:

Updates

Submission history