Whisper Town

What is Whisper Town?

Whisper Town is a real-time 3D village inhabited by 8 AI characters, each powered by a live LLM and given their own personality, memories, daily routine, and voice. They wake up in the morning, walk to work, gossip at the tavern, fall in love, worry about their bread rising, run for re-election, and head home when night falls — all without a single line of scripted dialogue.

The hero feature: whisper a rumor into any villager's ear, and watch the AI gossip spread organically through the village. Sam tells Tom. Tom reflects. Tom acts. The whole village shifts because of one secret.

It's not a chatbot. It's a society.

Inspiration

Most AI projects today are isolated chatbots — one model, one window, one conversation that ends the moment you close the tab. We wanted to build the opposite: a place where multiple AI minds live together, remember each other, and produce emergent stories nobody scripted.

The original spark came from Stanford's Generative Agents paper (Park et al., 2023), which simulated a small town of LLM characters. That work proved emergent social behavior was possible in text. Our question was: what happens when you put it in a living 3D world and let a stranger whisper rumors into it?

How it works

Every 20 seconds is one "tick" — an in-game hour. On each tick:

Perceive — the system gathers each villager's current location, who's near them, the weather, the time of day, and their recent memories.
Think — a structured prompt is sent to Gemini 2.5 Flash via a Supabase Edge Function. The model returns a JSON decision: a private thought, an action (move/talk/reflect/wait), and what to say or do.
Act — the villager walks, speaks (with synthesized voice), or reflects. Speech bubbles render above their head. Their TTS voice plays out loud.
Remember — the event becomes a memory in their personal store, available to influence every future tick.
Reflect — periodically, villagers compress recent memories into higher-level beliefs ("the town has felt heavier this week").

When the user whispers a rumor, it's injected as a new memory of type rumor with source "a stranger you trust". On the next tick, the villager's prompt naturally includes it — and gossipy villagers like Sam will start passing it on. Each retelling increments a hop_count so the spread is visible in the Town Feed as rumor · hop 1 → hop 2 → hop 3.

What I built

Real-time 3D village with day/night cycle, weather, rolling terrain, atmospheric fog, fireflies, and warm window glow at night
8 hand-crafted AI characters — Maya the anxious baker, Tom the lonely teacher, Sam the gossipy bartender, Elena the wise retired doctor, Jonas the lovesick blacksmith, Ruth the stressed mayor, Davi the dreamy music student, Priya the quiet herbalist
Live LLM-driven behavior via Gemini 2.5 Flash, called per villager per tick through a Supabase Edge Function
Persistent memory for every villager — they remember past conversations and reference them in future decisions
Whisper feature with a prominent global button — pick a villager, type a rumor, watch it spread
Voice synthesis via the browser's Web Speech API — each villager has a distinct pitch and speaking rate matching their personality
Town Feed showing every event in the village with type-coded color stripes and rumor hop tracking
Daily routine — villagers head home at night, work during the day, socialize in the evening
Interactive camera with full orbit/pan/zoom, click-to-select villagers
Side panel showing live current thoughts, recent memories, and current goals for any villager

Challenges I ran into

Building this in a no-code tool meant the path was not linear.

The Edge Function CORS wall — the LLM brain refused to connect from the browser for over 60 ticks. Tracing through HTTP error codes, we eventually found a missing x-client-info header in the Edge Function's Access-Control-Allow-Headers. One missing word in one allow-list.
Rate limiting — calling Gemini 8 times per tick blew past the free-tier quota. The fix was a rotating schedule: process only 2 villagers per tick in rotation, with retry-and-backoff on 429 responses.
Soft LLM rules don't stick — telling the model "please go home at night" was ignored half the time. The fix was a deterministic post-LLM override: if it's between 22:00 and 06:00 and the villager isn't at home, force the move regardless of what the model chose.
Memory persistence shaping behavior — the most magical moment was watching Priya's panel show "Jonas's whispers pull at me, but my purpose is to serve" — a memory of a love confession that visibly changed her later thoughts. That's when it stopped being a tech demo and became a story engine.

What I learned

That emergent storytelling is real and accessible — eight LLM calls per minute, plus a structured memory schema, is genuinely enough to produce believable social behavior.
That the demo-killer feature in any AI project is the moment the user feels causal. Watching a thought-bubble appear is interesting. Watching a rumor you planted reach a villager who reacts to it is unforgettable.
That constraints sharpen design. Gemini rate limits forced the rotating-villager pattern, which actually made the village feel more alive (always something happening, never all 8 deciding at once).
That no-code tools have a real ceiling, but it's higher than I thought. Real Three.js, real Edge Functions, real Gemini, all in MeDo.

What's next

Multi-room interiors — step into the tavern and see Sam pour drinks
Cross-village travel — rumors that move between separate towns via a wandering merchant
Player as villager — let the user be one of the AI agents and see how the others react to them over time
Save/share a town's history — bookmark a moment, share a permalink to your favorite rumor cascade