Inspiration
LEGO is one of those rare mediums where almost anything can become something you can build. But there is a huge gap between looking at an object and figuring out how to recreate it with LEGO bricks.
Ivwanted to explore a simple question:
What if you could take a picture of something in the real world and have a computer figure out how to build it out of LEGO?
That idea became Brickify, a system designed to transform a photograph into a buildable LEGO representation.
What We Built
Brickify is built around a multi-stage pipeline:
Photo → 3D Model → Voxels → LEGO Bricks → Parts List → Building Guide
Instead of trying to directly generate a LEGO model from pixels, we break the problem into smaller, more controllable stages.
First, an image is converted into a 3D mesh using AI-powered 3D reconstruction. The mesh is then normalized and converted into a voxel representation. From there, a procedural decomposition algorithm determines how the voxel volume can be covered using LEGO bricks.
Our current LEGO decomposition system supports multiple brick sizes, including 2×4, 2×3, 2×2, 1×4, 1×3, 1×2, and 1×1 bricks.
This approach gives us an intermediate representation that we can inspect, modify, and eventually use to generate the final LEGO model and construction instructions.
How We Built It
Brickify uses a React + Three.js frontend and a Python + FastAPI backend.
The backend handles the computational pipeline:
- 3D mesh loading and validation
- mesh normalization
- voxelization
- LEGO brick decomposition
- LEGO model data generation
The frontend is responsible for presenting the result interactively and will eventually allow users to upload an image, inspect the generated 3D model, view the LEGO reconstruction, and explore the generated building process.
We deliberately built the system incrementally rather than attempting to create the entire pipeline at once.
We first created a clean voxel representation for arbitrary OBJ and GLB meshes. We then developed a deterministic LEGO decomposition algorithm that attempts to cover each voxel layer with the largest possible supported brick before falling back to smaller bricks.
This modular approach means each stage can be tested independently before being connected to the next one.
Challenges
The biggest challenge is that 3D reconstruction and LEGO construction are fundamentally different problems.
An AI model can produce a visually plausible 3D object, but that does not automatically mean the object can be represented cleanly using LEGO bricks.
Voxelization gives us a useful bridge between these two worlds. Instead of trying to reason directly about arbitrary triangle meshes, Brickify converts geometry into a discrete grid that our LEGO algorithm can reason about.
Another challenge was making the decomposition deterministic. Given the same voxel model, we want Brickify to consistently produce the same LEGO arrangement. Our current algorithm processes voxels in a fixed order and uses a fixed brick preference order, making its output reproducible.
What We Learned
Building Brickify taught us that complicated AI applications don't necessarily need one giant AI system.
Breaking the problem into stages made the system much easier to reason about:
continuous 3D geometry → discrete voxels → discrete LEGO bricks
We also learned how important intermediate representations are when connecting different types of algorithms. The voxel grid acts as the common language between 3D reconstruction and LEGO construction.
What's Next
Our next major step is connecting AI-powered image-to-3D reconstruction to the existing mesh → voxel → LEGO pipeline.
From there, we plan to build the interactive experience around the pipeline, including:
- interactive 3D model visualization
- LEGO model visualization
- automatically generated parts lists
- step-by-step building instructions
- layer-by-layer exploration of the generated model
Our goal is for the final experience to feel almost magical:
Take a picture → watch Brickify build it out of LEGO.
Log in or sign up for Devpost to join the conversation.