AI House Helper

Inspiration

The idea came from a very real problem: I am starting to build a house. My own place to leave in like the thing that will stay there for the rest of my live.

I received information about my plot, terrain and ground conditions in professional drawings and documents. The information was technically there, but I could not easily explore it myself. Even a simple question—such as moving the house, changing a window, comparing layouts or planning the garden was just impossible i can draw it on a piece of paper but aren't the tomatoes too close to the house or to terrace i have no idea because i can't see the scale.

Traditional CAD understands geometry, but expects specialist knowledge. A conversational AI is easier to use, but normally cannot reliably understand or modify the real spatial model behind a drawing.

I built this project to connect those two worlds.

What it does

House WebMCP is a browser-based spatial editor where a person and an AI agent can work together on an early house, site and garden concept.

The application represents the project as structured building information rather than a decorative 3D model. It understands the site, terrain, buildings, storeys, slabs, rooms, shared walls, doors, windows, roofs, landscape zones, plants and garden fixtures.

A person can inspect and measure the project visually. Through WebMCP, an agent can read the same underlying model and help with requests such as:

Move the house and show me the result.
Add or resize a window on a specific wall.
Compare two possible layouts.
Create a raised-bed kitchen garden near the house.
Show the floor plans, elevations and sections.
Analyse seasonal conditions and planting considerations.

The agent never silently changes the project. Every proposed edit becomes a visible ghost variant. The user can inspect it, compare it with other alternatives, and explicitly apply or reject it. Committed changes can also be undone.

The bundled demonstration is grounded in my real house-building context in Zielonki. It combines site and terrain information with geotechnical evidence, an editable climate profile, soil unknowns and conservative planting guidance. It is intended for early exploration and better conversations with professionals—not as construction documentation or a replacement for an architect or engineer.

Why WebMCP

This use case needs more than an embedded chatbot.

Without WebMCP, an agent would have to interpret canvas pixels, guess what visual elements mean, or manipulate the page through fragile interface automation. It would not reliably know that an object is a shared wall, that a window belongs to that wall, or that changing a slab affects the spaces above and below it.

With WebMCP, the website exposes structured, validated operations over its actual application state. The agent can inspect the project, use semantic references, propose a precise change, display the result inside the existing interface and ask the person for approval.

This creates a form of collaboration that was previously difficult for someone without CAD experience: I can describe the outcome I want in ordinary language, while the application remains responsible for geometry, validation, visualization and project state.

The human provides intent and judgment. The agent translates that intent into structured actions. The application makes the consequences visible and keeps the final decision with the human.

How I built it

The application is written in React and TypeScript, with React Three Fiber and Three.js powering the interactive spatial view.

I created a semantic ProjectV2 model for buildings, storeys, shared slabs, wall graphs, polygonal spaces, hosted openings, roofs, platforms, landscape and planting objects. This model is shared by both the visual editor and the WebMCP tools, so agent actions use the same domain logic as human interactions.

The current application exposes 23 WebMCP tools. They cover project inspection, building and landscape edits, seasonal analysis, architectural views, variant comparison, approval, rejection and undo.

The tools are defined using Zod validation schemas. The same registry produces the runtime registrations, structured prompts and a searchable public tool manifest, helping prevent the documentation from drifting away from the implementation.

For geometry, Manifold runs in a Web Worker and creates slabs and walls with real door and window openings. Rapier provides constrained editing previews, while three-mesh-bvh accelerates spatial queries. Zustand manages application state and IndexedDB provides local autosaving.

Challenges I faced

The largest challenge was moving beyond a visually convincing 3D scene and creating a model an agent could safely reason about. A building is not just a collection of boxes: walls are shared by rooms, openings belong to walls, slabs can serve more than one storey, and changes must preserve those relationships.

Another challenge was designing safe agent interaction. Directly allowing an AI to mutate the committed model would be fast, but it would remove the user’s control. The ghost-variant workflow took more work, but it makes every agent action reviewable and reversible.

Working with real site information also required care. Surveyed facts, professional observations, working measurements and design assumptions do not have the same level of certainty. I learned to preserve those distinctions instead of turning incomplete evidence into false precision.

Finally, browser-based geometry can become computationally expensive. Moving Boolean geometry generation into a worker, caching results and carefully managing rendered resources were important for keeping the editor responsive.

What I learned

I learned that making an application agent-friendly is not simply a matter of adding tools to an existing interface. The underlying product needs clear domain concepts, explicit constraints and actions whose consequences can be inspected.

I also learned that WebMCP is most valuable when the agent and the person genuinely share the same workspace. The interesting part is not that an AI can call a function. It is that it can understand live application state, propose meaningful work, show the result and return control to the person.

Most importantly, building this project changed how I think about my own house-building process. I may still not know CAD, but I can now explore ideas, understand trade-offs and prepare better questions before speaking with architects and other specialists.

What's next

The next steps are to support controlled import of professional survey and building data, improve planning and environmental checks, and make it easier to share reviewed variants with architects, landscape designers and engineers.

The long-term goal is not to replace those professionals. It is to give homeowners a clearer, safer and more accessible way to participate in the decisions that shape their future home.