Globe Gen

• 

  Example continuous terrain

• 

  Start

• 

  Fill horizontal bridges

• 

  Fill vertical bridges

• 

  Fill holes

Inspiration I wanted to build something for video games that could help 3D game developers generate terrains in continuous 3D worlds. In this kind of world, if you move toward one side of the map, you wrap around to the other side instead of hitting a boundary.

Given my passion for machine learning and computer vision, I wondered if I could adapt machine learning techniques to create maps that loop around instead of being bounded. Most game worlds are bounded, but enabling a map to be continuous opens up interesting possibilities for gameplay and exploration. I specifically focused on the terrain generation aspect of this idea, using real Earth elevation data to ground the system in geographic structure.

What it does Sand Sifter is a terrain generation system for 3D grid-based worlds that expands landscapes using masked heightmap inpainting. Instead of generating terrain in a single pass, it incrementally “marches” across a grid, conditioning each new tile on partially observed neighbors and filling missing regions through a learned generator trained on real-world elevation data from multiple continents.

The system is designed for continuous-world game environments, where terrain wraps or extends seamlessly across boundaries. It uses partial masks and overlapping context windows to enforce local consistency while a blending stage smooths out tile seams, producing coherent large-scale terrain without visible grid artifacts.

In addition, it constrains the terrain to wrap around itself by allowing the heightmaps to be unbounded. The terrain size can be modified, and once terrain is generated, it can be saved as a PNG file that stores the heights at different coordinates.

How we built it During training, random spatial masks are applied to simulate missing regions, and the model learns to fill in the missing regions from partial observations. For terrain generation, the model applies this technique starting from a random half image from the dataset and expands and bridges unfilled regions by tile marching, taking into account the fact that the world has no boundaries.

The user can also change the terrain size to generate smaller or larger worlds as needed. Once the terrain is generated, it can be exported as a PNG file, where each pixel encodes elevation values at different coordinates, allowing the result to be saved, shared, or reused in other tools or game engines.

Challenges we ran into The biggest challenge was maintaining consistency across tile boundaries. While individual tiles looked realistic, maintaining seamless continuity was challenging. It required careful iteration. There were multiple scenarios where the tile marching algorithm would attempt to fill a hole from one side without accounting for other filled regions next to the hole.

Training the model was also very difficult and required a large and complex architecture that could both encode the known heights and generate a heightmap that contained both the known regions and creatively filled in the unknown regions.

Accomplishments that we're proud of I am proud of how my models turned out and how well they trained despite limited computing resources. I am also very proud of how smooth and visually consistent the terrain transitions look, especially given the challenges of maintaining continuity across generated regions.

What we learned One of the key insights I obtained from the project is that local realism does not equate to global consistency. Even though the model worked well on the patches it was trained on, using the model to bridge generated patches was a fundamentally harder problem.

I also learned how small modifications to the model can have a big impact on its success. This made me realize how sensitive generative systems are to seemingly minor design choices.

Another important takeaway is that iterative generation systems introduce compounding challenges, where small design flaws can cause the system to completely fail.

I also gained a lot of knowledge about utilizing computer vision for complex tasks. Although I originally intended not to use machine learning in the project, I am fascinated by how computer vision models were applied to expand terrains.

What's next for Sand Sifter I intend to train even more complex models for the task of terrain generation with a more direct goal in mind. I think that would improve how seamless the transitions are. Currently, the model is trained to bridge gaps only from real-world data and not generated data, even though during execution we encounter scenarios where the engine must fill a hole between two generated tiles.

Built With

• python
• pytorch
• ursina