CanvasMind

A floor plan an AI agent can actually read; audit for wheelchair and other access.

Comment

Inspiration

Generally, Canva is used for planning venue layouts, and most of time those designers are not accessibility auditors. Thus, one day someone in a wheelchair finds he cannot reach a table, let's say table six, or a fire exit – which perhaps is behind the buffet.

The reason is nobody checks properly, because checking properly means measuring every gap and tracing every route from every door. That's tedious arithmetic by hand and genuinely impossible by eye , which is exactly the kind of work you'd want to hand to an AI.

However, it’s not that simple. And before you think it's a matter of model quality, it’s not. Rather, the reason behind the inefficiency of AI here is poor structural understanding.

What it does

CanvasMind is a room planner that tells you whether a wheelchair user can reach every seat.

You drag furniture, resize it by its corner handles, name the room. It checks four things continuously:

  • furniture outside the room
  • overlapping items
  • the 150cm clear zone in front of every doorway
  • whether every item can actually be reached from a door along a continuous 90cm-wide route

That last one is the hard one, and it's what pairwise gap checks miss: furniture spaced perfectly but walled off, with no route to it.

The verdict is stated in human terms — "8 of 31 seats unreachable", not "8 rule violations" — and you can export the plan as an image with that verdict stamped on it, so the file is a record you can send a client.

And you can ask an AI agent to do any of it.

How we built it

Vanilla JavaScript, the Canvas 2D API, and WebMCP. Zero dependencies, no build step, no framework — about 1,400 lines across three modules and one HTML file, deployed as static files.

Deliberately no canvas library. SVG-based options (React Flow, D3-as-SVG, draw.io) render a readable element tree, which would defeat the entire premise. Canvas 2D emits pixels and nothing else.

18 tools registered via document.modelContext.registerTool(), every one pointing at the same functions the mouse handlers call. One implementation, two callers — a person dragging furniture and an agent calling a tool run identical code, so they cannot drift apart.

  • readOnlyHint splits the surface. The five audit tools (get_layout, find_items, describe_selection, measure_gap, check_accessibility) are marked read-only so an agent can investigate freely; the twelve that mutate the room aren't.
  • A live tool list. fix_accessibility is registered with an AbortController passed as the registration signal; aborting that signal unregisters it. The toolchange event drives the tool count in the UI: 17 → 18 → 17.
  • AbortSignal honoured inside a running tool. auto_arrange animates a force simulation across many frames and stops on the frame when cancelled.
  • Descriptions written for a model, not a human. Units stated ("all units are centimetres, (0,0) is the top-left"), preconditions stated, and guidance on when to call. That prose is the real interface, and most of the tuning went there.
  • Provenance. Each item carries a dot showing whether a person placed it, the agent placed it, or the agent moved it.

The geometry the model never has to do

The agent decides to call check_accessibility. The page does the maths:

  1. rasterise the room at 10cm resolution
  2. mark cells occupied by furniture
  3. erode the free space by the 90cm wheelchair footprint
  4. flood-fill from each doorway, walking inward and stopping at the first obstruction — so a table parked in an opening makes that door unusable rather than being stepped over
  5. any item with no reachable cell touching it fails

fix_accessibility repacks the room into rows with real aisles built in, so the result is compliant by construction rather than by luck, then relocates anything still boxed in.

That split is the point: the model contributes intent, the page contributes domain expertise.

Challenges we ran into

Chrome and the spec disagree about inputSchema. Chrome's getTools() returns it as a JSON string while the W3C CG draft types it as an object. That double-encoded every schema in my UI until I normalised both. Chrome's executeTool() argument shape also diverges from the draft, so the built-in tool runner invokes registered functions directly rather than round-tripping through the consumer API. The agent path is unaffected — but the divergence cost real debugging time.

Correct spacing does not imply reachability. My first repair implementation shoved furniture apart until every pairwise gap passed the 90cm standard. Every gap was legal and only 1 in 6 randomised layouts was actually accessible, because the packing left walled-off pockets. I replaced it with deterministic row packing that builds aisles in, so accessibility holds by construction. Now 6 of 6.

Erosion arithmetic. Using ceil to size the erosion kernel demanded ~110cm of clearance to certify a 90cm route — so a corridor built exactly to standard failed its own check. One character, completely wrong answers.

The flood fill stepped over blocked doorways. It seeded by searching outward from a door until it found any free floor, which meant it jumped straight over a table parked in the opening and declared the room reachable. Fixed by walking inward from the door and stopping at the first obstruction.

Finding the right problem. The first version was a generic node-graph editor that proved the WebMCP point perfectly and helped nobody. The canvas argument was right; the domain was wrong. Reskinning it to accessibility auditing kept every line of the engine and gave it an actual audience.

Accomplishments that we're proud of

The reachability engine is genuinely correct, and I can prove it. It catches furniture that is perfectly spaced but walled off, and doorways that are technically clear but blocked in practice. 25 automated checks cover it, including a 20-layout randomised invariant that no repair ever pushes an item through a wall.

The tool list is alive. Watching fix_accessibility appear when the room breaks and withdraw itself when it's repaired is the clearest demonstration I could build of something a static MCP server structurally cannot do.

It degrades honestly. With no WebMCP at all — plain Chrome, Safari, Firefox — the planner still works by mouse or keyboard, and the built-in Tool runner panel invokes every tool directly. The whole project can be evaluated with zero setup.

An accessibility tool that is itself accessible. Tab cycles items, arrow keys move the selection 10cm, Shift+arrow moves 1cm, R rotates, Delete removes, Esc deselects. It would have been embarrassing to ship an accessibility auditor that required a mouse.

What we learned

WebMCP's value isn't the plumbing — it's the app declaring what it means. An agent can guess your structure from your markup, and that guess is what browser actuation already is. What it can't do is derive intent. A registered tool is a contract the site author wrote; a scraped one is a guess an outsider made.

Tool descriptions are the real interface. No validator checks them, and they determine whether the model picks the right tool. Stating units, preconditions, and when-to-call did more for reliability than any schema tightening.

The model should contribute intent, not computation. Every time I was tempted to let the model reason about coordinates, the right answer was to give the page a better solver and let the model just ask. It's also why this beats a vision agent: the app already owned the spatial reasoning and only needed a way to be asked.

Building on a moving spec means testing against the implementation, not the document. Two of my four real bugs came from trusting the CG draft over what Chrome actually ships.

What's next for CanvasMind

  • Import real floor plans — DXF or an image with a scale reference, so you audit an actual venue instead of a rectangle.
  • Multi-room buildings with routes between them, including corridors and lifts. Reachability across a whole building is the same algorithm at a larger scale.
  • Configurable thresholds per jurisdiction. The current figures follow the spirit of ADA / BS 8300 guidance; the numbers should be selectable, not baked in.
  • A shareable audit link, so a venue and a client can look at the same plan and the same verdict.
  • More than wheelchairs — turning circles, guide-dog space, step-free routes, seating sightlines.

This is a demonstration project, not a certified compliance tool, and the roadmap is about earning that distinction.

Built With

Share this project:

Updates

Submission history