AI Driven Simulation Engine

• 

  Choose a simulation or define your own. Everything starts here.

• 

  Experience the scene. Hear the narration, see the world, and feel the moment unfold.

• 

  Choose an action, or write your own. The simulation responds to what you decide.

• 

  Your skills and inventory evolve with your actions, just like in real life. The simulation remembers every decision

• 

  When a story resolves, the system generates a final video representing its outcome.

• 

  Bad decisions can end the simulation early. Choose carefully.

• 

  The AI offers subtle hints when you’re stuck, but never forces a path.

Inspiration

This project is inspired by old text-based computer games, where everything happened through words. Those games felt incredible at the time, but they were hard to play. You had to remember commands, follow strict rules, and stay inside a very narrow path.

I always loved the stories, but wished the worlds could react more freely.

With today’s AI, that is finally possible.

AI-Driven Simulation Engine brings that idea back in a modern way. Stories are created as you play, actions are not limited to commands, and the world reacts naturally to what you do, step by step.

What it does

AI-Driven Simulation Engine is a first-person AI simulation where you step into a world and act freely inside it.

You can start from a pre-generated scenario or define your own concept. From there, the engine takes control of the narrative, the atmosphere, and the evolution of the world itself.

Your actions shape who you become inside the simulation. Skills grow or decline based on behavior. Inventory changes depending on what you do and how you do it. The story follows your decisions, not a script.

If your actions align with the logic of the world, the simulation can resolve into a meaningful ending, rewarded with a final cinematic sequence generated as video. If your decisions are careless or reckless, the simulation can reach a bad outcome and simply stop. Choices matter.

How I built it

The system is built around Gemini as a narrative and reasoning engine, orchestrating every part of the simulation in real time.

Gemini 3 Flash is used to generate the narrative quickly and consistently, keeping latency low even across long runs. Each turn produces structured outputs that define the state of the world, including story progression, skills, inventory, and available actions.

Images are generated to visualize each scene, text-to-speech brings the narration to life, and background music is created to reinforce atmosphere. For completed story arcs, video generation is used to produce a final ending sequence.

Gemini is also used as a lightweight agent inside the system. When structured outputs cannot be reliably parsed, Gemini is invoked as a lightweight validation agent to correct and normalize skills, inventory, and state changes according to the rules of the current simulation before they are applied.

As simulations grow longer, Gemini also generates compact story summaries, allowing the world to keep evolving without losing coherence over time.

Challenges I ran into

The first challenge was freedom.

Allowing players to act freely without breaking the world would traditionally require massive amounts of handcrafted logic. By using Gemini as a narrative director, the system can interpret open-ended actions and evolve the simulation naturally instead of relying on rigid paths.

This required careful prompt design and strict structured outputs working together. Each simulation can introduce different skills, items, and narrative rules, so it was critical to not only generate structured data, but also validate it. Gemini is also used as a lightweight validation agent. After generating structured outputs, it checks that skills, inventory, and state changes follow the rules of the current simulation before they are accepted.

The second challenge was latency.

Generating narrative, images, audio, and sometimes video in a single turn risks breaking immersion. Using fast reasoning models and parallel generation was essential to keep the experience fluid and responsive.

Accomplishments that I'm proud of

The level of immersion achieved through multimodality. Each simulation feels alive, combining story, visuals, sound, and progression into a single experience.

I'm also proud of the freedom the system allows. Users are not choosing from fixed options, they are expressing intent. The simulation adapts, reacts, and continues forward, even in unexpected directions.

What I learned

This project reshaped how I think about applications built with AI.

Instead of tools that wait for prompts, I can build systems that evolve, reason over time, and respond to human decisions in a meaningful way. Experiences that feel less like software and more like worlds you enter.

AI-Driven Simulation Engine feels like a glimpse of how future applications might work, not just answering questions, but participating in long-running experiences shaped by human choice.

---

What's next for AI Driven Simulation Engine

The next step is deeper immersion.

Real-time conversations with characters, evolving music that reacts dynamically, richer video sequences, and voice-driven interaction instead of typing.

I also want to expand beyond fiction into realistic, historical, and forward-looking simulations.

Experiences where understanding the past, exploring complex systems, or testing future scenarios is safer and more valuable inside a simulation than in the real world. Stepping into those environments, observing consequences, and learning from decisions before they happen is what drives this project forward.

Built With

• fastapi
• gemini
• gemini-3-flash
• javascript
• python