From Computer-Aided Design to AI-Aided Design

Inspiration

CAD has been computer-aided for fifty years, and the computer's role hasn't changed much. Every question about a design still gets answered by a human clicking through it, and every problem found that way gets typed into a form somewhere away from the model.

I've spent years building the browser viewer that does the clicking-through. WebMCP made me wonder what happens when the viewer stops being a display and becomes a set of tools an agent can use. Not AI that generates a design, but AI that can review one.

What it does

A BIM design review app: a Revit model in the Autodesk Platform Services Viewer, with the viewer and the review's issue list exposed to an agent as WebMCP tools. You walk the building in first person, select something, and talk.

The agent searches design metadata, browses the model hierarchy, isolates and frames parts, colors objects by any property, etc. Then it drafts an issue against it, revises the draft while you watch, and submits when you say so. show-issue puts you back inside the model at the reporter's eye position, with the viewer state they had.

It reads the current selection and camera every turn, so "what is this door?" resolves without either of us naming anything.

How I built it

• APS Viewer loads and renders the translated Revit model, and provides the first-person walkthrough.
• WebMCP registers several tools straight from the page, with return values shaped for a token budget rather than for a database client.
• The form is the draft. draft-issue writes into the same form you can type into. submit-issue takes no arguments at all. The only thing that can be filed is what's on screen.
• Tools register conditionally on viewer state: none until geometry has loaded, and all ten dropped before a different design loads. A property query against a half-loaded model doesn't give you a degraded answer, it gives you a confident wrong one.
• Nothing leaves the browser. Issues go to IndexedDB. "The north stair doesn't meet code" next to a client's project name is a professional judgement about someone else's building, and a hackathon demo has no user auth to protect it with.

What I learned

The interesting work wasn't wiring up tools, it was deciding what a tool should say back. A property query on a real model returns thousands of rows, which is useless to an agent. Returning counts, outliers and a category breakdown is what turns guessing into answering. My favourite question in the demo, "how many doors have no fire rating set?", is one no amount of looking at the screen can answer, and it costs about twenty tokens.

The best tool I wrote was the one I deleted. I built a screenshot tool so the agent could check its own camera moves. It worked, and then I confirmed the host reads the viewport directly without being asked.

A viewpoint beats a picture anyway. Half the problems worth reporting are only visible because a floor was isolated or the model was cut at mid-height. Store the camera alone and the reviewer arrives at the right coordinates staring at an opaque wall, which is worse than not navigating at all, because it makes the issue look wrong.

Challenges

• Shared state. Two actors driving one camera needs a clear owner. Every agent move is an animated flight rather than a cut, because a cut drops you somewhere else in the building with nothing to say how the two places relate.
• Trust. If submit-issue accepted overrides, the review step would be a formality. The agent says "I set it to high and submitted it" and you have no way to check. Forcing every edit back through the on-screen form is what makes "looks good, submit" mean something.
• Scale. Engineering models have hundreds of thousands of objects. Every list is capped and carries its true total, and color-coding paints recursively, so 5,000 doors cost one id.
• Deixis. Making "this" work meant treating the viewport as context the agent always has, rather than something it asks about.

Built With

• aps-viewer
• autodesk-platform-services
• bun
• render
• typescript