Plumb — the agent-first floor-plan CAD

Inspiration

Floor plans are where a renovation, a rental listing, or a new office starts — but CAD tools are built for experts, and AI chatbots can't draw a straight wall. We wanted the best of both: a real drafting canvas where an AI space-planner works beside you like a colleague with a red pen. It reads your plan, checks it, and proposes fixes you approve with one click — and when you give it a blank canvas and a sentence, it drafts the room itself. Owning the whole thing on open infrastructure (open models, open formats, open license) is what the Nebius × NVIDIA hackathon is about, so we built Plumb there from day one.

What it does

Plumb is an agent-first 2D floor-plan and interior-design app that runs in the browser:

• Draft — draw walls, doors, windows, and furnish from a 19-piece catalogue on a Canvas2D plan (two-bed demo: 70.0 m², 5 rooms, 24 pieces).
• Check — the agent scans every door swing (exact analytic OBB-vs-quarter-disc test), overlap (SAT on oriented boxes), room containment, 900 mm passage clearance, and window blocking, and explains each issue in plain language.
• Propose, don't impose — every fix appears as a dashed red-pen ghost on the canvas. The agent can only propose; applying is always your click, with one-click undo and a re-check follow-up.
• Create from nothing — "Create a single empty 6 metre by 4 metre room with one door and one window" produces 8 validated ops: room, 4 walls, door, window, zero conflicts (shown live in the demo).
• Interchange — one click exports a clean DXF (AIA layers, door/window/furniture blocks, real dimensions, A3 1:50 paper space, 0 audit errors) and IFC4 (8 walls, 11 openings, 4 doors, 7 windows, 5 spaces with finishes). DXF files import back, with the agent reconstructing walls, doors, and rooms from raw geometry.
• Everything is an Op through one validated pipeline, so user edits, agent edits, and imports share undo/redo and review for free.

How we built it

Stack: React 18 + TypeScript (strict) + Vite frontend, hand-written Canvas2D renderer (React never touches the drawing loop), Zustand store, Zod-validated op system, Vitest + Playwright, Python ezdxf conversion service, and a Hono/Node agent server.

Nemotron on Nebius Token Factory is the agent's brain. Every agent turn runs on nvidia/Nemotron-3_5-Lightning served through Token Factory's OpenAI-compatible /chat/completions:

• The server streams an agentic tool loop over SSE — get_plan → run_checks → find_space → propose_edit → done — with up to 16 rounds and an 8k output budget for whole-plan builds.
• propose_edit dry-runs every op batch and returns the issues that would remain, so Nemotron sees its own mistakes and self-corrects before you ever see a proposal (visible in the demo thread: invalid batches get retried, then land clean).
• We benchmarked both Token Factory catalog models live: Lightning answered tool calls in ~1–2 s with ~130–460 completion tokens, vs ~2–5 s and 3–7× tokens for the 120B Super-VL — so the fast text2text model drives every interaction, with Super-VL reserved for a future plan-image mode.
• The API key never leaves the server; the browser only sees redacted /api/health (provider: nebius, model: nvidia/Nemotron-3_5-Lightning). A deterministic regex fallback agent keeps the app fully usable with no key, no network, and in every test.

Main feats we're proudest of: the analytic door-swing check replacing sampled math with identical results; an op system where wall-remove cascades and undo restores exactly; a hand-written IFC4 exporter (909 entities) with no native dependencies; a DXF round-trip that recovers the semantic model exactly via XDATA; and a 2:17 demo film — Playwright-captured app footage, Deepgram voiceover, HyperFrames assembly — where every number on screen was measured, not mocked.

Challenges we ran into

• The model guessed op shapes and burned rounds on schema errors — fixed by documenting literal op templates in the tool description, so first-try batches validate.
• Truncated reasoning — a 1024-token cap cut whole-plan batches mid-JSON; raising to 8192 plus tolerant parsing (JSON-string batches, single-op wraps) made multi-room builds reliable.
• Silent 401s — a truncated API key failed with a bare "model call failed"; error hints now surface status + provider message (never the key).
• Timeout mismatch — the client aborted at 35 s while the server budgeted 150 s, killing long builds mid-stream; both now agree at 150 s.
• No CAD stack in the environment — installed ezdxf into the convert service live to prove the 0-audit-error export; ifcopenshell wouldn't fit, so IFC is hand-written STEP (validated by entity/reference tests).
• A full disk once made every tool write empty files — diagnosed via exit codes, cleared caches, kept building.

Accomplishments that we're proud of

• A working agent loop on Nemotron-3.5-Lightning: live tool calls, dry-run validation, self-correction, and from-scratch drafting — all on open infrastructure.
• Design-grade output: zero-audit-error DXF that opens in LibreCAD/AutoCAD, IFC with real spaces and quantities.
• An agent with manners: proposals, ghosts, Apply/Dismiss/Undo — the human always decides.
• Proof, not slides: the demo shows the checks, the proposal cards, the export layer tables, and the health endpoint — every figure reproducible from the repo.

What we learned

• A small fast model beats a big one for tool use: Lightning's latency and token discipline made the agent feel instant; reasoning models spent thousands of hidden tokens to emit the same ops.
• Dry-run validation is the whole ballgame — letting the model see issuesAfter before the user sees anything turned a flaky demo into a reliable one.
• Open formats are a feature: XDATA-tagged DXF and semantic IFC mean a Plumb plan survives outside Plumb.
• Deterministic fallbacks aren't just for tests — they're what keep an agent app feeling alive when the network doesn't.

What's next for Plumb

• Room polygons from the wall graph (grid flood-fill prototype works; Clipper2 union next) replacing hand-set rects, with name/finish carry-over.
• DWG support slotted in front of the DXF importer as an external converter step.
• Plan images to Super-VL — the multimodal path our benchmarks reserved.
• 1:50 PDF export, Nebius Serverless deployment for the convert service, and Yjs multiplayer on top of the op log (designed for it already).
• Bigger catalogue, multi-storey, and a mobile layout pass.

Built With

• nemotron
• vite