Inspiration
Everyone has seen a chess game where one move is so bad the whole room goes quiet. We wanted to make that moment the fun part.
Most chess tools show scores, best-move arrows, and commentary. We took the opposite approach: a hidden engine watches the game, memes and sound effects react to it, you can play with your voice, and Gemini reads you a Gen Z recap at the end.
What It Does
- Live Multiplayer: Play chess with a friend by sharing a room link, or use pass & play on one device.
- Voice-Controlled Chess: Just say your move and the pieces move. Illegal moves are ignored, so a wrong guess never breaks the game.
- Hidden Stockfish Engine: Stockfish analyzes every move in the background. Players never see scores or hints.
- Meme Reactions: Reactions pop up based on what actually happened, like a hung queen or a lucky escape. Each player gets their own reaction.
- Fun Sound Effects: Goofy noises play alongside the reactions, so a blunder sounds as bad as it looks. Sound is on by default and can be muted using the speaker button or settings.
- Gen Z Gemini Summary: Gemini writes a playful recap of the game in Gen Z slang, covering the blunders, comebacks, and moments everyone will want to laugh about again.
- Text-to-Speech: The summary is read out loud, so you can listen to your game get roasted.
How We Built It
We used TypeScript, React, Vite, Node.js, and Socket.IO. Stockfish runs as a WASM engine in a separate Node process, and the Gemini API generates the summaries. Sound effects are synthesized directly in the browser, so there are no audio files to download.
The server holds the chess state, validates every move, and runs the engine. After each move, it compares the position before and after:
$$ \Delta = e_{\text{after}} - e_{\text{before}} $$
A big evaluation swing against the player who moved triggers a "blunder" reaction. Only the chosen reaction is sent to the browser, so evaluation scores never reach the client.
Voice with ElevenLabs
ElevenLabs handles both directions of speech. Its speech-to-text converts what a player says into a chess move, which then follows the same path as clicks and drags.
The server validates every spoken move against the rules, and illegal moves are simply dropped.
AI-Powered Game Summaries
For the summary, the server sends the game's moves and flagged moments to the Gemini API and asks it to generate a short, funny recap in Gen Z slang.
ElevenLabs text-to-speech then reads the result aloud. All API keys stay securely on the server.
Challenges
- Timing: Engine analysis takes time, so obvious events like captures and checks trigger reactions instantly, while engine-based reactions follow.
- Stale Results: Players can move faster than the engine can analyze. We tag every position with a version and discard outdated results.
- Flickering Reactions: Near an evaluation boundary, reactions could flip back and forth. We implemented separate enter and exit thresholds to prevent this.
- Finding the Right Meme: Matching the right meme to each situation was challenging, since a blunder, a lucky escape, and a close call all call for different reactions.
- Voice Moves: Speech can be unclear or describe an illegal move, so the server validates every spoken move and ignores invalid ones.
- Setup Limitations:
npm installfailed in our build environment, so we tested the reaction logic independently first.
What We Learned
- How to run a chess engine as a managed background process.
- Why chess evaluations require careful handling: perspective matters, and mate scores aren't ordinary numbers.
- How to build a multiplayer server that controls the game state while keeping the client uninformed.
- How to combine an engine with an LLM, with Stockfish supplying the facts and Gemini telling the story.
- How to integrate voice input and output using ElevenLabs, allowing players to interact with the game without touching the board.
- How small touches like synthesized sound effects and slang can make a game significantly more entertaining.
What's Next
- Make the memes animated using real GIFs.
- Deploy publicly with HTTPS.
- Add chess clocks, user accounts, and improved touch controls.
Built With
- elevenlabs
- geminiapi
- node.js
- react
- socket.io
- stockfish
- typescript
- vite
- webaudioapi