Inspiration
Livestreaming has traditionally been a one-way medium: the streamer plays the game, and the chat watches, comments, and occasionally spams emotes. We wanted to fundamentally flip this dynamic by making the audience the game. Inspired by interactive Twitch chat phenomena and community-driven storytelling games, we set out to build a platform where thousands of viewers can actively collaborate, debate, and vote on branching narrative paths in real-time. We wanted to bridge live chat interactions with a structured, AI-driven roleplaying backend so that every broadcast becomes a unique, unpredictable, and shared community adventure.
What it does
CrowdQuest AI is a Twitch-connected collaborative quest platform that transforms any live stream into an interactive roleplaying game.
Real-Time Chat Commands: Viewers influence the unfolding story directly through Twitch chat using commands like !quest, !vote 1/2/3, !help, !stats, and !leaderboard.
Deterministic State Machine: Manages voting rounds with first-vote-wins enforcement, tie policies, single-contribution limits per viewer, and transactional XP rewards to keep gameplay fair and orderly.
AI Quest Composer: Integrates with Ollama (or OpenAI-compatible endpoints) using strict Pydantic validation to dynamically generate immersive story prompts and path options.
Streamer Dashboard & Live Controls: Gives broadcasters a central command center to start quests, monitor live metrics, and manage session flows.
Secure OBS Browser Overlay: Features a revocable, private token-secured browser source with live WebSocket updates so viewers watching the stream can see voting stats and path choices in real-time.
How we built it
We architected CrowdQuest AI using a modern, robust tech stack designed for speed, scale, and local-first reliability:
Backend & State Engine: Built with Python, FastAPI, and SQLAlchemy/Alembic, backed by a PostgreSQL database for persistent state and Redis for rate-limiting, pub/sub, and real-time fan-out.
Twitch Integration: Leveraged the Twitch OAuth authorization-code flow with encrypted token storage, a dedicated Twitch EventSub WebSocket client for channel.chat.message, and Helix chat messaging.
Frontend & UI: Developed using Next.js, React, TypeScript, Vite, and Tailwind CSS to deliver a polished, responsive streamer dashboard and overlay views.
AI Runtime: Powered by local-first open-weights models via Ollama (such as Qwen models) with strict JSON schema validation and deterministic content fallbacks to ensure uninterrupted storytelling if API latency spikes.
Challenges we ran into
EventSub Reliability & Deduplication: Handling high-velocity Twitch chat streams without dropping votes or getting rate-limited required implementing robust Redis-backed deduplication and structured consumer worker queues.
State Synchronization: Ensuring that vote deadlines, pause/resume clock preservation, and WebSocket fan-outs stayed perfectly synchronized between the backend state machine, the streamer dashboard, and the OBS overlay without race conditions.
AI Output Consistency: LLMs can occasionally drift or produce malformed JSON schemas. Enforcing strict Pydantic parsing alongside deterministic fallback mechanisms ensured the game state never crashed mid-stream.
Accomplishments that we're proud of
End-to-End Twitch Integration: Successfully implementing a secure OAuth token exchange pipeline, EventSub WebSocket client, and Helix chat response system from scratch.
Deterministic Fairness: Designing a bulletproof voting and contribution engine that enforces first-vote-wins rules, tie-breaking policies, and one-contribution-per-viewer limits cleanly.
Production-Ready Architecture: Shipping a containerized solution complete with Docker Compose orchestration, Alembic database migrations, comprehensive unit tests, and a secure OBS overlay system.
What we learned
The importance of robust asynchronous workers and rate-limiting when interfacing with high-throughput third-party real-time APIs like Twitch EventSub.
How combining strict deterministic state machines with generative AI provides the best of both worlds: creative, infinite storytelling anchored by reliable, predictable game progression rules.
What's next for CrowdQuest AI
Distributed Broadcaster Scaling: Implementing multi-worker Redis leasing to support horizontal scaling across thousands of concurrent live channels.
Advanced Visual Generative Elements: Integrating on-the-fly image and soundscape generation so that winning vote paths dynamically render custom scene illustrations directly onto the OBS overlay.
Expanded Game Systems: Introducing persistent inventory systems, character classes, and cross-stream seasonal campaigns for communities.
Built With
- alembic
- docker
- docker-compose
- fastapi
- next.js
- ollama
- postgresql
- pydantic
- python
- react
- redis
- sqlalchemy
- tailwind-css
- twitch
- twitch-eventsub
- twitch-helix
- typescript
- vite
