Grounded - Collaborative AI Workspace for Construction

What I built

I built Grounded because construction document review mixes work that agents handle well with judgments they should not make alone. An agent can search schedules and compare written requirements. Reading plan symbols, following linework, or counting openings on a drawing still needs a construction professional to do reliably.

The tools are meant to work in a lot of review setups, but most people do not have contract drawings sitting around, so the demo ships with a fixed project. It includes an open-source Virginia Farmhouse drawing set and two example submittals. The door package is seeded with real issues for the agent to find. The window package is clean on purpose, so the review has a known pass case next to a known fail.

What the agent actually gets is structured. It can search the evidence we already extracted from the PDFs, jump the workbench to a page, block, or region, and open an Assistance Request when it hits something it should not guess at. The human answers in the same workspace. The agent comes back later and finishes the disposition from that structured response.

Why WebMCP fits

These workflows are a mess of strengths and weak spots. Agents are good at finding textual evidence across a pile of sources and stitching it together. They still struggle at reading construction drawings the way the job requires. AEC-Bench is a useful look at how hard that still is. I treat drawings as close to a worst case for today's agents, and the cost of a miss is high. A denied submittal can add weeks. A bid off by 50% can be hundreds of thousands of dollars.

So I stopped asking the model to do the part a construction professional already does well, and I stopped asking that professional to do the part the model does well: collecting and reconciling evidence across a pile of documents. WebMCP is what makes that split practical. Without it you usually get a painful loop. The agent asks for something, you open another app, mark up the drawing, export it, and paste answers back into chat. With WebMCP the agent discovers tools on the live page, works from immutable document versions, and creates a request tied to exact pages. The reviewer answers in place. After a reload the agent can recover the response without depending on conversation memory.

I do not think this pattern is limited to construction. Anywhere an agent needs a durable human judgment tied to a specific artifact, the same shape applies.

How I built it

Grounded registers eight WebMCP tools in one all-or-nothing attempt and only reports "WebMCP ready" when the full set is up:

get_grounded_help, get_project_workspace, list_project_documents, search_project_documents, inspect_document_evidence, navigate_document, create_assistance_request, get_assistance_request

The document tools expose prepared artifacts. There is no runtime OCR and no external document-processing call in the demo path. Search can point at likely content, but a claim has to come from inspected evidence. Navigation moves the visible PDF workbench to a page, a semantic block, or a normalized region for visual inspection or to direct the user where they should be looking.

Assistance Requests are the durable handoff. For a Point Set, the reviewer clicks locations on the PDF. Marks live in an overlay separate from the PDF.js display layer so they stay aligned through resize and zoom. Responses persist in IndexedDB under a tab-scoped Demo Session. Unfinished Zustand drafts intentionally die on reload. Committed responses do not.

Contracts use TypeBox for TypeScript inference, browser registration, and runtime validation. A recording Model Context adapter exercises the same behavior under Vitest. Stack is React 19, TypeScript, Vite, Dexie, Zustand, a custom PDF.js display layer, and Cloudflare Workers with static assets via Wrangler.

Challenges I ran into

Getting the agent to stop inventing visual counts was the hard product problem. Early versions would "see" openings that were never marked. The fix was process, not prompting: search is a hint, inspect is required, and drawing interpretation goes to the human through a Point Set instead of a freeform chat answer.

The other hard part was teaching the agent how to use the app without a skill. Normal MCP is starting to grow discoverable skills that come with the server. WebMCP has something drawn up for that, but it is early, and I did not want to hand judges a big skill install just to run the demo. So a lot of the work went into tool names, descriptions, and contracts that could carry the workflow on their own. That ended up working, but a skill would have made the same behavior clearer, and on a real project I would almost certainly put one on the agent so it knows how to work with our documents.

I also wanted a much larger tool set than the eight that shipped. Context made that a bad idea. These reviews chew through a lot of the agent's window. A lot of it is cached across repeated tool calls, which can get better, but the structured evidence coming out of the document parser was still token-heavy and full of fields that did not help the decision. We had to thin that payload until the loop fit. It is in a decent spot now, but still could be better.

What I learned

The real benefit of WebMCP, for me, is collaboration. I went into this hoping an agent could own more of the review than it can. It cannot. Not yet. Drawing interpretation, accountable counts, and the judgment call behind a disposition still need a person who knows the work.

That does not make the collaboration temporary. Even as the models get better at reading drawings, a shared workspace like this stays useful. The human still needs to stay current on what the agent found, what it skipped, and what it is asking for. WebMCP is a clean way to keep that loop in one place instead of exporting screenshots and pasting answers back into chat.

I do not see WebMCP as a stopgap. Once you have a durable way for an agent and a professional to hand work back and forth against the same artifact, a lot of other workflows open up. Construction is just the one I built first.

What's next

Widen the Assistance response types past Point Sets, support real project imports beyond the bundled Virginia Farmhouse set, and harden the review trail so a disposition can cite every mark and every document version it used. Longer term I want this sitting inside an actual submittal workflow, not only a hackathon demo session.

Built With

• cloudflare
• dexie
• indexeddb
• pdf.js
• react
• typebox
• typescript
• vite
• webmcp
• zustand