Dwell AI: Hackathon Submission Materials

🚀 Elevator Pitch (Limit: 200 Characters)

Dwell AI democratizes interior design. Transform 2D floor plans into shoppable 3D dream homes instantly. Don't just imagine your space see it, refine it, and shop the look with confidence.

💡 Inspiration

The genesis of Dwell AI was a simple observation: Gemini's image editing capabilities are incredible, but they often lack context. We asked ourselves, "What if we could harness that power for surgical edits in interior design?" Instead of just generating a pretty picture, we wanted to build a system that understands the room allowing users to manipulate furniture as easily as moving a layer in Photoshop, but with the realism of a photograph. We realized that true design requires more than generation; it requires precision.

🏠 What it does

Dwell AI is an intelligent interior design agent that solves the "blank canvas" problem. Users upload a photo of a floor plan or an empty room, and Dwell AI:

1. Understand: Analyzes the space geometries using Computer Vision to create a digital twin.
2. Design: Generates multiple creative layout options based on architectural principles (e.g., maximizing natural light).
3. Visualize: Renders the result in photorealistic 3D, allowing users to "feel" the space.
4. Refine: Lets users chat with the design (e.g., "swap the sofa for a sectional") to make instant updates.
5. Shop: Finds real furniture products that match the generated design.

⚙️ How we built it

We architected Dwell AI as a multi-agent system using LangGraph, where each agent is powered by a specialized Google Gemini model:

• Vision Agent (Gemini 3 Pro): Extracts walls, windows, and doors into a coordinate-based spatial graph.
• Designer Agent (Gemini 2.5 Pro): The "brain" that reasons about layout constraints and placement rules.
• Perspective Agent (Gemini 2.5 Flash): The "renderer" that generates the visuals.
• Shopping Agent (Multimodal): Executes visual search queries via SerpAPI. The frontend is built with Next.js 16 and Tailwind CSS 4, while the backend runs on FastAPI.

🚧 Challenges we ran into

Building a bridge between 2D pixels and 3D spatial understanding was incredibly difficult.

1. Object Detection: Accurately identifying existing furniture and architectural elements from a raw photo was a constant battle against noise and perspective distortion.
2. Precision Movement: Translating a user's intent (e.g., "move the bed to the wall") into precise coordinate updates for the layout engine required complex geometry math.
3. The Hardest Part: 2D Perspective Generation: Generating a consistent 2D side-view perspective from a top-down floor plan was our biggest hurdle. We had to teach the model to "hallucinate" the vertical dimension (height, depth) accurately while respecting the strict 2D floor plan constraints.

🏆 Accomplishments that we're proud of

We are most proud of the "Surgical Edit" capability. Seeing the AI successfully move a specific piece of furniture without distorting the rest of the room felt like magic. We also successfully integrated a full shoppable loop, proving that AI can drive real-world commerce, not just inspiration.

🧠 What we learned

We learned that context is king. A powerful model like Gemini 3 becomes exponentially more useful when you ground it in structured data (our spatial graph). We also discovered that "constraints" actually make the AI more creative by giving it boundaries (walls, doors), it produces far more realistic and usable designs than an unconstrained image generator.

🔮 What's next for Dwell AI

• AR Integration: Projecting the generated designs directly into the user's physical room.
• Material Swapping: allowing users to change flooring and wall textures in real-time.
• Partnerships: Integration with major furniture retailers for direct checkout.

Built With

• axios
• fastapi
• gemini-3-pro-preview
• git
• google-gemini-2.5-flash-image
• google-gemini-2.5-pro
• google-generative-ai-api
• httpx
• konva
• langgraph
• langsmith
• lucide-react
• nano-banana
• next.js-16
• pillow
• pydantic
• python
• radix-ui
• react-19
• react-dropzone
• react-hot-toast
• react-konva
• serpapi
• shapely
• tailwind-css-4
• typescript