Marbles Race AI

An autonomous, multi-agent AI system where Gemini designs, tests, heals, and evolves physics-based marble race worlds daily—combining procedural generation, self-verification, real-time commentary.

Comment

Inspiration

We were inspired by a simple question: what happens when AI is not just generating content, but is responsible for maintaining a living system over time?

Marble races are deceptively simple, yet they combine physics, unpredictability, competition, and spectacle. We saw an opportunity to turn this into a testbed for autonomous AI orchestration—where AI does not merely respond to prompts, but designs, verifies, repairs, and evolves an entire world daily without human intervention.

Our goal was to move beyond “prompt-based apps” and build a self-governing AI system.

What this project does

Marbles Race (MarbleUniverse) is a physics-based 3D marble racing world that rebuilds itself every day.

Each day:

  • AI agents design a brand-new race track
  • The track is simulated, tested, and healed automatically
  • Races are run with real physics
  • Events are narrated live
  • Results are woven into an evolving mythology shared with the community

The system is fully autonomous after deployment.

How we built it

The project is structured as a multi-agent system orchestrated with Gemini, running across frontend, physics simulation, and backend automation.

🧠 Gemini Agents

  1. The Historian
    Finds a positive historical event matching the current date and proposes a thematic concept for the track.

  2. The Architect
    Translates the concept into a strict JSON blueprint describing 3D spatial segments (straights, curves, inclinations), ensuring physical plausibility.

  3. QA Sentinel (Self-Healing)
    Runs headless “phantom races” using real physics simulation to detect failures such as marbles falling, getting stuck, or exceeding safe speeds.
    If issues are found, the Architect is forced to regenerate the track until it passes validation.

  4. The Artist
    Generates cinematic cover art using gemini-2.5-flash-image, aligned with the day’s theme and difficulty.

  5. The Track Brain (Marathon Agent)
    A long-running agent that remembers past tracks, performance metrics, and player behavior.
    It influences future tracks by adjusting difficulty, structure, and risk based on historical outcomes.

  6. The Myth Engine
    Connects race results, historical inspiration, and community statistics into daily stories and newsletters, published automatically.

Autonomous verification and self-correction

A key design goal was AI that verifies its own work.

Instead of trusting generated content:

  • Every track is simulated multiple times
  • Physics telemetry (falls, DNFs, speeds, collisions) is collected
  • Failure reports are converted into natural language feedback
  • The system iterates until safety and playability thresholds are met

This creates a self-healing feedback loop, not a one-shot generation.

Real-time interaction and learning

During races:

  • A hybrid commentary system reacts instantly to events using local telemetry
  • Gemini provides higher-level analysis and narrative continuity
  • The system understands cause and effect, not just outcomes (e.g. why a crash happened, not just that it happened)

Over time, the system learns what makes tracks exciting, fair, and memorable.

Technologies used

  • Gemini 3 – multi-agent reasoning, long-running orchestration, validation, narrative generation
  • Three.js – real-time 3D rendering
  • Rapier (WASM) – deterministic physics simulation
  • Supabase – database, auth, storage, scheduled Edge Functions
  • Resend – automated daily newsletters
  • React + App Router + Tailwind – frontend experience

Challenges we faced

  • Designing track geometry that is both visually expressive and physically stable
  • Aligning AI-generated layouts with real physics constraints
  • Preventing cascading failures in procedural generation
  • Creating agents that could disagree, retry, and converge instead of blindly accepting outputs
  • Maintaining continuity across days using long-running memory

These challenges pushed us to treat AI as an engineering system, not a content generator.

What we learned

  • Autonomy emerges from feedback loops, not prompts
  • Verification is more important than creativity in generative systems
  • Long-running agents change how you think about application design
  • Physics-based environments are powerful tools for testing AI reasoning

What’s next

Marbles Race is designed as a foundation for:

  • Evolving marble genetics and strategies
  • Player-personalized coaching agents
  • Deeper cause-and-effect analysis of complex simulations
  • Autonomous creative worlds that grow over time

This project is not a demo—it is a living AI system.

Built With

Share this project:

Updates