Dream Catcher

Inspiration

The concept of our project was Oasis, which is a fully AI generated Minecraft clone, where each frame is generated procedurally as the user plays. We wanted to build off the idea of creating a game where every play-through is a completely new game.

For the theme and graphics, we were inspired by the look of old PlayStation 1 and DS style horror games. We also drew inspiration from the game Paradise Chateau. We loved its unsettling atmosphere, and the feeling that these retro-style games bring.

What it does

Dream Catcher is not a horror game. It is a horror game generation framework. When you press "New game" a completely unique experience is generated.

It starts with a concept of a well documented haunting or legend, then uses and LLM to build a storyboard. The storyboard is then converted to a structured JSON output (also using an LLM) that is used to build the game.

The piece that brings our project to life is our custom parser. This parser defines a contract that allows the LLM to generate output that can converted to a fully playable game. This contract defines all the constraints for ways that rooms, items, obstacles, and NPCs can be created. It also defines how these entities behave, interact, and progress the story.

The framework also generates all assets from scratch, ensuring that every part of the experience is uniquely generated for the player.

How we built it

To build this project, we first analyzed the atomic components that make up games of this style. Some of these include:

Rooms - individual contained scenes with doors that connect to other rooms.
Static obstacles - things like walls, rocks, trees, tables, etc.
Intractable objects - items such as books, chests, etc.
Items - things that the player can pick up and hold in their inventory.
NPCs - both hostile and friendly, mainly for plot progression.
Jump scares :P

For each one of these, we created properties to these components. Some of these include:

Transformations - position, rotation, scale of an object.
Collisions - defines how an objects can collide with other objects, also exposes on-collision events (ex. colliding with a hostile NPC takes damage).
Emission - allows objects to produce other objects (ex. interaction with an NPC causes the NPC to emit a key, or changes the objective).
Textures - defines how an object looks and how they animate.

Our engine uses TypeScript, Three.js, and Rapier for rendering, gameplay, and physics. We use Claude Opus 5.5 to generate game content, the open-weight Qwen-Image model for artwork, and ElevenLabs for audio.

Game generation

LLMs handles game generation in two stages.

First, it uses an in-depth, well tuned prompt to generate a storyboard for the game. The story board defines the plot, the map, and a high level overview of how different rooms should work together to create a cohesive game.

Then we make a separate prompt is used to generate the structured output for each room, all running in parallel. This prompt tells the LLM how to convert part of the storyboard into the structured output, explaining in depth the possible decisions the LLM should make, and what to avoid. Each call returns JSON describing the room's geometry, lighting, props, characters, dialogue, keys, and floor sounds.

Our code connects the rooms and assigns their IDs. This keeps the LLM from having to manage references across dozens of separate calls.

We also wrote a parser and validator to catch broken rooms before they reach the player. It uses an LLM and an additional prompt to check for overlapping geometry, requires paths to be at least 1.2 meters wide, and prevents enemies from spawning too close to the player. If a room fails validation, the LLM repairs the room for up to two attempts.

Art and animation

After the game generation is completed, the structured output contains a textures array, that gives a texture id and a prompt that can be passed to an image model to generate each texture.

Nano Banana is used to generate the pixel art for each game, including tileable wall and floor textures, furniture, props, and character sprites.

For animations, we feed the first frame back as a reference when generating the remaining frames to keep the character consistent.

To generate transparent textures, we generate sprites against a green background and run a cleanup pass to get rid of green outlines around the edges.

Our rendering pipeline runs the game at one-third of the screen resolution, then applies 4x4 Bayer dithering and limits colors to 16 shades per channel. This gives the game its desired PS1-horror look.

Audio and gameplay

ElevenLabs generates monster screams based on descriptions written by the LLM during game generation. Each scream takes roughly two seconds to generate. We also use it for the menu theme.

The rest of the audio runs through our Web Audio system. It combines recorded footsteps with synthesized wind, rain, dripping water, and a heartbeat that gets faster when enemies approach.

Characters use Rapier's kinematic character controllers to chase, wander, or patrol their rooms.

Deployment

We deployed the game on Google Cloud Run with a Node.js server. Generated games are stored in a Google Cloud Storage bucket mounted into the container, and API keys are managed through Secret Manager.

Challenges we ran into

Getting AI-generated rooms to work

LLMs could generate convincing room descriptions, but that didn't mean the rooms were playable. We ran into significant issues when attempting to generate fully cohesive games. Some of these issues include keys spawning inside walls, objects overlapping, and enemies appearing directly on top of the player. We fixed these issue through pure trial and error, tuning the prompt after each generation until the games became cohesive and playable.

We spent a large part of development defining the JSON format, validating the generated content, and building the repair system. The game needed to be able to reject a bad room without breaking the entire house.

Generating over 100 images per game

Our Eilean Mòr game alone needed 128 images. At that scale, using Nano Banana became expensive, which led us to consider open-weight models such as Qwen-Image and FLUX since these could be run on a dedicated GPU, which for our use case was significantly cheaper.

Unfortunately, these models struggled to adhere to prompts, and generated poor quality assets.

We solved this problem by reverting back to using Nano Banana and tuning our game generation prompts to encourage reusing assets when possible. We saw a nearly 30%-40% reduction in image generation usage with no noticeable visual difference.

Accomplishments that we're proud of

We got Dream Catcher to achieve generate a playable horror game from a single button press. The player doesn't need to write a prompt, design a level, or provide any assets.

Our room generation and validation system can assemble connected levels with keys, doors, dialogue, enemies, and custom artwork.

We're also happy with how the game looks. The output exceeds a much higher level of quality than we expected would be possible.

The jump-scares also managed to scare us during testing, despite knowing exactly how they worked.

Most importantly, the game is deployed and playable through Cloud Run. Generated houses are saved so people can revisit them.

What we learned

The biggest lesson was how much structure matters when generating game content.

Early on, we focused on getting better output from the LLM. Eventually, we found that defining a strict game format and handling IDs, geometry checks, and errors in code made a bigger difference than changing prompts.

We also learned to design around generation time. Generating over 100 images takes a while, so we built a loading screen that shows the story, rooms, and art being created. Prioritizing the starting room also lets people play before the rest of the house is finished.

Finally, working within the PS1 style helped us deal with inconsistencies between generated assets. Low resolution, limited colors, dithering, and fog made separately generated textures and sprites look more consistent.