Inspiration
Twitch chat is great at making decisions together—but what if those decisions could teach an AI?
We wanted to turn reinforcement learning into something people could understand by playing. Our starting point was a simple question: What happens when an AI learns exactly what you reward, even when that isn’t what you intended?
MEDUSA grew from that question into a community competition. Viewers become coaches, creatures develop unexpected habits, and every match reveals the consequences of chat’s choices.
What it does
MEDUSA — Reward League is a 3D AI coaching game built for Twitch streamers and their communities.
Viewers join Team Mint or Team Coral through Twitch chat and vote on the rewards used to train their creature. They can reward scoring, touching the ball, making progress toward the goal, or avoiding mistakes.
Those choices change what the AI learns. Reward ball touches too generously, and a creature can learn to collect points while barely scoring. Reward the intended outcome, and its behavior can improve.
The experience includes:
- Eight arenas: Explore coral reefs, changing tides, gravity shifts, solar winds, wormholes, and the deep seabed.
- Team competition: Creatures attack opposite goals on independently simulated, mirrored courses.
- Two-change rematches: Communities can repair a disappointing strategy by changing only two reward components.
- Streamer Derby: Two streamers can connect their communities to a shared hosted match.
- Temptation Round: Chat chooses between productive incentives and tempting alternatives.
- Captain’s Dilemma: Streamers choose environmental conditions that challenge both teams. The central interaction is simple: chat teaches what matters, then watches what the creature actually learns.
How we built it
We built MEDUSA with OpenAI Codex as a development collaborator, iterating from an initial reward-design experiment into a deployed multiplayer coaching experience.
The learning system uses tabular Q-learning in a discrete game environment. Each team trains for 10,000 episodes using its selected reward recipe. We then replay the learned policy so viewers can see the resulting behavior.
Three.js renders the creatures, floating pitches, and themed environments. GSAP supports animations and visual transitions. The 3D presentation follows the underlying simulation, keeping the learning system separate from its visual effects.
Twitch OAuth and EventSub connect viewer chat commands to team selection, coaching votes, and predictions. Shared derby rooms coordinate the host and guest experience through a backend API.
We published the source on GitHub and deployed the application on Render.
Codex helped us implement the learning loop, build arena mechanics, integrate Twitch, develop shared rooms, investigate bugs, and prepare deployment configuration and automated checks.
Challenges we ran into
One challenge was making the learning understandable. A high reward total can look like success even when a creature fails to score. We needed to show game outcomes alongside reward totals so viewers could recognize that difference.
Fair competition also required care. We used mirrored courses, equal training budgets, and comparable starting conditions. Rematches preserve the kickoff conditions, helping players assess the effect of their reward changes.
Another challenge was translating a discrete simulation into an appealing 3D experience. Creature movement, camera positioning, effects, and readable feedback had to communicate what was happening without hiding the underlying behavior.
Finally, connecting Twitch participation, voting windows, shared rooms, and deployment required careful coordination between browser state and backend state.
Accomplishments that we're proud of
We turned reinforcement learning into an interaction that people can understand without reading an equation: choose incentives, watch behavior, and try again.
We’re especially proud of:
- Making reward exploitation a visible part of gameplay.
- Connecting Twitch chat to meaningful training decisions.
- Building eight environments with different tactical conditions.
- Creating a constrained rematch that encourages thoughtful experimentation.
- Supporting collaboration between streamers through shared derby rooms.
- Shipping a playable public deployment with an accessible source repository.
What we learned
An agent can become very good at earning rewards while remaining bad at the task we care about.
Making that distinction visible became the heart of MEDUSA. Players need to see both what they rewarded and what actually happened.
We also learned that audience participation becomes more interesting when decisions have lasting consequences. Coaching a creature creates anticipation: viewers want to discover whether their strategy worked—and debate how to repair it.
Building with Codex reinforced the value of small iterations, explicit requirements, and verification. Each new arena or community mechanic needed to work with the learning system and contribute to a clear player experience.
What's next for Medusa
We want to develop MEDUSA into a recurring community event with persistent teams, saved reward recipes, and tournament progression.
Our next priorities are:
- Improving onboarding and feedback for first-time coaches.
- Adding clearer comparisons between original and repaired strategies.
- Expanding arena mechanics and testing how policies adapt.
- Strengthening room persistence and reconnect behavior.
- Improving synchronized broadcast overlays.
- Exploring richer learning methods and new community-designed challenges. Our goal is to make learning itself a spectator sport—where the audience helps create the strategy, discovers its flaws, and earns the comeback.
Built With
- javascript
- openai
- reinforcement
- three.js
- twitch
