LayoutLens

LayoutLens turns plain English into a 3D room visualization. Runs on local LLMs or hosted APIs. Creates room boundaries, doors/windows, and object placements in minutes.

Comment

Inspiration

As a game developer and 3D artist, I kept running into a familiar pain point: I can describe an environment I'd like to create in words, but turning that into a concrete layout takes a long time (sketching, measuring, moving things around). For the hackathon, I wanted a fast loop where I could type a prompt and immediately see a rough 3D layout that is good enough to iterate on.

What it does

LayoutLens takes a natural-language prompt and produces a structured JSON room design: A space (room boundary, ceiling height, doors/windows) A room plan (placed elements with position, rotation, footprint, and height)

It validates the geometry of the layout, writes the JSON data to an output folder, then an Unreal visualizer loads the JSON and spawns placeholder meshes so you can see the layout in 3D.

How we built it

  • Python + PydanticAI agents generate the design, space, and final plan.
  • A geometry service validates the output and triggers retries with clear error messages.
  • Outputs are written into per-run folders under ./output/ so every run is saved.
  • A custom Unreal Engine 5.7 C++ plugin reads room_plan.json, spawns simple boxes for floor elements, and draws walls/openings from the room boundary.

Challenges we ran into

  • The biggest issue was collisions/overlaps. Even when the JSON was valid, small overlaps would cause retries and wasted iterations.
  • Giving good “move this object” suggestions is harder than it sounds.

Accomplishments that we're proud of

  • The system is resilient: it saves JSON data every run and can recover from bad outputs instead of failing silently.
  • The Unreal side is simple but effective: walls + openings + labeled placeholders make the result readable fast.

What we learned

  • Strong schema + validation makes LLM outputs dramatically more usable.
  • Retries only help if the error messages are actionable—good validation feedback matters.

What's next for LayoutLens

  • Better room creation: generate the room boundary from a simple text/ASCII sketch (or a tiny “draw on grid” UI).
  • Better geometry: add lightweight packing rules (bigger “keep-out” margins, fixed walkways, zone-based placement).
  • Real meshes: replace placeholder boxes with real assets through AI mesh generation (via modern text/image-to-3D pipelines such as Hunyuan3D).
  • Multi-room support: extend Space + RoomPlan to support multiple rooms and connecting doors.

Built With

Share this project:

Updates