Dream House

Inspiration

Interior designers often spend countless hours transforming 2D floor plans into 3D visualizations, a process that’s both time-consuming and technically demanding. Mastery of CAD or 3D modeling software is often required just to visualize a design. Dream House was created to streamline this workflow. It enables designers to sketch, iterate, and generate immersive 3D interiors within minutes, freeing them to focus on creativity rather than technical overhead.

What it does

Dream House converts rough 2D floor plan sketches into detailed, editable 3D spaces:

The user begins by sketching a floor plan.
The system refines it into a clean, formal layout.
After customization and adjustments, the finalized floor plan is rendered as a fully furnished 3D environment, all at the press of a button.

How we built it

The frontend was developed with Next.js, while the backend runs on FastAPI.

The initial sketch is processed by Nano Banana to generate a formalized floor plan.
The refined plan is passed through SAM (Segment Anything Model) to classify and extract furniture coordinates and dimensions.
CubiCasa5K is used to detect and map walls.
Using Trellis, we generated a repository of 3D furniture assets. Each detected furniture item is matched to the closest 3D model, which is then rendered in Unity alongside the extracted wall geometry to produce the final scene.

Challenges we ran into

The toughest challenge was achieving reliable extraction of furniture and wall layouts from floor plans. We experimented with Nano Banana masking, coordinate mapping, and YOLOv3 before settling on a hybrid approach using SAM for furniture segmentation and CubiCasa5K for wall detection. Integration across multiple frameworks also required careful pipeline design to maintain data consistency and accuracy.

Accomplishments that we're proud of

We’re proud of building a robust, end-to-end system that integrates computer vision, generative modeling, and real-time 3D rendering into a single cohesive workflow. Despite numerous technical hurdles and integration challenges, Dream House consistently produces accurate, realistic 3D representations from simple sketches.

What we learned

This project taught us how to coordinate multiple complex tools and ML models within a unified architecture. We learned the importance of detailed planning and system design before implementation, as early oversights often led to rework and communication gaps. Most importantly, we experienced the power of combining AI and visualization technologies to simplify creative workflows.

What's next for Dream House

Next, we plan to:

Improve segmentation accuracy by fine-tuning or training domain-specific models for furniture recognition.
Expand our 3D furniture library with more customizable assets.
Add advanced interior customization features, such as materials, lighting, and decor variation.
Deploy Dream House as a web platform for interior designers to explore, iterate, and visualize designs effortlessly.