I took a lot of inspiration from https://www.jackboxgames.com/, a game series which I love playing with my friends!
Plot is a room-based multiplayer AI-video generation game. Create a lobby with your friends and we take headshots of your face from 3 angles to create a realistic animated figure. Watch the main screen for prompts and use your phone to lock in your responses and build your adventure!
How we built it
I built plot.surf as two connected interfaces: a host screen for the TV and a controller on each player's phone. Both use React and Vite. They stay in sync over WebSockets with a TypeScript backend built on Node and Fastify, which runs the room, the question timers, the votes and the video pipeline.
To turn players into cartoon characters, we used Nano Banana 2 on fal. It converts each selfie into a stylized portrait. It then draws every scene using each character's portrait and photos as reference images, so everyone still looks like themselves from shot to shot.
Claude Sonnet 5.5 is the writers' room. From the group's answers, it writes the episode title, the outline, and for each beat the narration, the image prompt and the motion prompt. It returns structured JSON validated against schemas. It also quietly swaps out any answer that crosses a line.
MiniMax Hailuo on fal animates each still into a 768p clip. ElevenLabs Eleven v4 voices both the narrator and the game-show host, with audio tags for dramatic delivery. Its word-level timings let us sync TikTok-style captions to the narration.
Challenges we ran into
The biggest challenge was time. Generating video takes far longer than a party will wait, and players expect their episode the moment the last vote closes. So we built the episode while people were still playing. Each answer starts its part of the video as soon as it's locked in. While the final vote is still open, we write and voice every possible plot twist in advance and start drawing the current leader's scene. We also wrote a priority job queue so the scene that plays first always gets the next free slot on our limited fal account.
The other challenge was making sure the show never breaks. Any AI call can be slow, fail, or refuse a player's answer, so every step has a deadline and a fallback. If a clip isn't ready, we show an animated panel made from the still image, and if that isn't ready either, we fall back to pre-made art of the location with narration. If Claude is slow, template scripts fill in. We also logged a timing trace for every game, which showed us exactly which steps were holding up the episode.
Log in or sign up for Devpost to join the conversation.