W O R L D S

Running becomes the story.

Comment

Inspiration

Zombies, Run! showed us that people are willing to run for a story. But we wanted to go further than placing a linear audio experience on top of a workout.

We asked: what if running itself controlled the story?

What if turning left chose your route, speeding up helped you escape, and being slow cost you a comrade's life?

That idea became Worlds: an audio adventure where the world is authored, but the experience adapts to how you run.

What it does

Worlds is an audio-first interactive running game where the runner becomes a character inside the story.

Our playable world, Greywatch: Blood in the Chapel, places the runner in the middle of a tense conflict between three characters. Instead of choosing from a traditional menu, the runner makes decisions through movement:

  • Turn left or right to choose a path
  • Maintain a certain pace, or if you fail to do so, there are consequences
  • Speak naturally to answer characters

The story remembers these decisions, changes what happens next, and ends with a recap of the path the runner created.

How we built it

We built the mobile experience using Expo, React Native, React, and TypeScript. It includes story discovery, run setup, dialogue playback, live pace and distance displays, movement decisions, speech input, captions, and a post-run recap.

The backend was built with Node.js, Express, and TypeScript. It manages the authored story graph, each runner's decision history, movement outcomes, generated scenes, and audio delivery.

Gemini becomes a constrained game director. It receives the runner's previous decisions, character emotions, target pace, and the rules of the world. It then generates a structured continuation without contradicting established characters or consequences.

ElevenLabs gives each character a distinct voice and performs newly generated dialogue with the emotion selected by Gemini. We also used speech recognition so runners can answer characters naturally instead of pressing a button if the local device's speech-to-text fails.

Challenges we ran into

The hardest challenge was creating a story that was immediately understandable while still making the player care about its characters. Every line had to establish the conflict, reveal personality, and lead naturally into a running interaction without feeling like a fitness tutorial.

Creating believable voices was another major challenge. We experimented with ElevenLabs voices, emotional direction, pacing, and dialogue length to make conversations feel like scenes rather than ordinary text-to-speech.

We also had to decide how much of the story should be authored and how much should be generated. Fully generated stories could become inconsistent, while fully prerecorded stories could not react deeply to the runner. Our solution was a hybrid system: writers control the world and its major consequences, while Gemini directs the live experience within those boundaries.

Finally, coordinating branching logic, audio playback, speech recognition, API latency, and fallback states inside one seamless experience required significant testing.

Accomplishments that we're proud of

We are most proud that the story feels like more than a technical demonstration. Players become invested in the conflict and begin forming attachments to Mara, Garrick, and Edric before making decisions that determine what happens to them.

We also built the complete interaction loop:

  • A fully voiced branching story
  • Decisions controlled through running
  • Consequences that persist across scenes
  • Natural spoken responses
  • AI-generated continuation that respects the authored world
  • Emotionally expressive ElevenLabs performances
  • A visual recap of the runner's unique journey

Most importantly, the AI is not an extra feature placed on top of Worlds. It directly enables the central experience: a story that listens and responds to the runner.

What we learned

We learned how to use ElevenLabs to create distinct characters, direct emotional delivery, generate dialogue dynamically, and integrate voice into an interactive mobile experience.

We also learned that generative AI works best when it has strong creative boundaries. Gemini produced more believable scenes when it acted as a director inside an authored world rather than being asked to invent everything freely.

Finally, we learned how important timing is in audio-first design. Dialogue length, pauses, transitions, recognition latency, and fallback behaviour can completely change whether a scene feels immersive.

What's next for Worlds

The next step is connecting Worlds to live GPS, pace, route, and motion data so every interaction can happen automatically during a real outdoor run.

We also want to introduce:

  • Persistent campaigns that continue across multiple runs
  • Stories that adapt to the runner's available time and distance
  • Creator tools for building new interactive worlds
  • Wearable integration and fully hands-free controls
  • Cooperative stories shaped by multiple runners
  • Collaborations with authors, studios, and established IPs

We imagine runners entering officially licensed fantasy, science-fiction, mystery, and superhero worlds, with their real movement determining how each adventure unfolds.

Worlds is not a story you listen to while running. It is a story you run.

Built With

Share this project:

Updates

Submission history