Genesis

Overview of the game UI
Character creation interface

Inspiration Every tabletop RPG group needs a great Game Master, someone who can narrate vivid stories, voice unique characters, draw maps, manage complex rules, and adapt to unpredictable player choices, all simultaneously and in real-time. Great GMs are rare. Most groups never find one, and millions of people who'd love tabletop RPGs never get to play because there's nobody to run the game.

We asked: what if AI could do all of that at once, not just generate text, but illustrate every scene, voice every character differently, compose the soundtrack, and manage a persistent world that remembers everything? Not a chatbot with a fantasy skin, but a true creative director that thinks in multiple modalities simultaneously.

What it does Genesis is a cinematic, multiplayer tabletop RPG powered by a 6-agent AI Game Master built on Gemini and Google ADK. Players create characters, join from anywhere via invite link, and the AI runs the entire game:

Narrates vivid prose that adapts to player choices in real-time.

Illustrates every major scene using Gemini's native interleaved output (text + images in one API call).

Voices each NPC with a distinct personality — the gruff dwarf sounds nothing like the noble elf (9 Cloud TTS voice profiles).

Composes procedural ambient music that shifts from peaceful tavern to epic battle dynamically.

Manages D&D-style combat with initiative, dice rolls, attack resolution, and XP progression.

Remembers everything; NPCs track interactions with sentiment, factions shift allegiances, and player choices ripple through the world as consequences.

Supports multiplayer with D&D-style action windows where all players declare actions, then the AI narrates everything together cohesively.

How we built it Backend: Python FastAPI server on Cloud Run with Google ADK orchestrating 6 specialised Gemini agents (Narrator, Rules, Art Director, World Keeper, Video Producer, Sound Designer). Each agent has its own tools and system instructions. A handler registry pattern routes 17 different tool calls to focused handler functions.

Multimodal Pipeline: Gemini 2.5 Pro generates narration with response_modalities=["TEXT", "IMAGE"] for true interleaved output. Imagen 3 handles portraits and battle maps. Veo 2 generates cinematic cutscenes. Cloud TTS provides 9 distinct voice profiles. A procedural Web Audio API engine generates ambient soundscapes.

Multiplayer: WebSocket connections per session with an ActionWindow system; exploration uses 12-second collection windows (like a real DM asking "what does everyone do?"), and combat uses strict initiative-based turns. WebRTC provides peer-to-peer voice chat between players.

Living World: NPC memory with sentiment tracking, faction reputation system, consequence engine for ripple effects, keyword-triggered lorebook, D&D 5e XP progression with auto-tracked achievements, and AI-generated loot with rarity and lore.

Infrastructure: Fully automated, one shell script provisions all Google Cloud resources. Cloud Build auto-deploys on every push to main. 70 automated tests across 5 layers.

Challenges we ran into ADK session management was our biggest surprise. InMemorySessionService requires explicit session creation before run_async, which isn't clearly documented. Sessions silently fail without it.

Vertex AI model names differ from the Gemini API names (imagen-3.0-generate-002 not imagen-4). We burnt several deploy cycles discovering this through Cloud Run logs.

Multiplayer concurrency was harder than expected. When 3 players type simultaneously, 3 AI calls fire and produce conflicting narrations. We solved this with the action window pattern, batching all player actions into one coherent prompt, which is actually how real D&D works.

Cloud Run + WebSockets requires --session-affinity for multiplayer to work; without it, different players' WebSocket connections route to different instances and can't see each other's game state.

CORS with credentials; allow_credentials=True is incompatible with allow_origins=["*"] in FastAPI's CORS middleware. It took several deploy cycles to diagnose the silent OPTIONS 400 errors.

Accomplishments that we're proud of True interleaved multimodal output: Not "generate text, then separately call an image API," but Gemini producing narration and illustrations in a single response stream. This is the creative director experience the competition asked for.

The action window system: It elegantly solves the multiplayer chaos problem by mimicking how a real DM operates: "everyone declare your actions... okay, here's what happens." One AI call, one cohesive narration covering all players.

NPC memory that actually works: Simple event + sentiment tuples create surprisingly believable NPC behaviour. The bartender you saved genuinely treats you differently. The guard you insulted remembers.

Production quality at scale: 70 tests, clean architecture (no file over 422 lines), thread-safe multiplayer, graceful fallback chains, auto-save, input validation, and one-command infrastructure provisioning. This isn't a hackathon prototype; it's built to run.

The full sensory experience: Text, images, distinct character voices, ambient music, and video cutscenes all flowing together in real-time. When it all clicks, it genuinely feels like playing a video game RPG, but with infinite creative freedom.

What we learned Multi-agent > single prompt. Separating narration, rules, art, and world management into specialised ADK agents produced dramatically better output than a single monolithic prompt. Each agent focuses on what it does best.

Graceful degradation is essential. Every media generation can fail — Imagen quota, TTS timeout, Veo unavailable. Building fallback chains (interleaved → Imagen → text-only) means the game never breaks, it just gracefully loses one modality.

The AI context window is the real game engine. By injecting NPC memories, faction standings, active consequences, lorebook entries, weather effects, and player backstories into every AI call, the model produces coherent, personalised storytelling without any hardcoded story logic.

Real-time multiplayer needs coordination, not just broadcasting. Simply sending all messages to all clients creates chaos. The action window pattern — collect, batch, process, broadcast — is the key insight that makes multiplayer AI narration work.

What's next for Genesis Gemini Live API integration: Real-time voice conversation with NPCs, where you literally talk to the characters and they respond with their unique voice, including interruption handling.

Interactive dungeon maps: Tile-based explorable maps with fog of war, token movement, and dynamic room generation as the party explores.

Vector database for NPC memory: The current list-based approach works for sessions, but a vector DB would enable NPCs to recall relevant memories across entire multi-session campaigns.

Spectator voting: Non-players watch live sessions and vote on "fate interventions" ("Should the merchant betray the party?") that the AI weaves into the narrative.

Mobile companion app: A simplified player view for phones, so you can play Genesis from anywhere with just a character sheet and voice input.