BoardRoom

Banner
Main tab
PCB tab
Errors tab

Inspiration

A missed decoupling capacitor or an I²C bus with no pull-ups turns into a three-week board respin and a few hundred euros. Single-LLM "PCB reviewers" miss real defects and invent fake ones with equal confidence.

But a real design review was never one expert. It's a board of specialists who disagree and have to justify themselves. So I built that instead of another chatbot.

What it does

BoardRoom runs a PCB review as a society of agents:

Five specialists — power integrity, signal integrity, connectivity/ERC, DFM & layout, and firmware bring-up — inspect a real KiCad project through the KiCad MCP server. Each agent can only touch the tools in its own lane, enforced in code, not by prompting.
Every finding must cite a real tool call. Uncited claims are rejected at the schema boundary, so the hallucination guard is architectural rather than a polite request in a prompt.
A Moderator detects conflicting recommendations and runs a bounded two-round debate — one extra measurement per side — then rules on the evidence, recording which measurement decided it.
The report is a blast-radius graph (findings → nets → components), plus board overlays drawn by a multimodal model that actually sees the rendered PCB.

How I built it

Qwen on Alibaba Cloud Model Studio, routed by role: qwen3-max chairs and adjudicates, qwen-flash runs the tool-heavy specialists, qwen3-vl is the multimodal layout critic, qwen3-coder generates bring-up artifacts.
26 typed KiCad MCP tool adapters as the specialists' only path to the board, behind per-agent allowlists enforced before any call.
Evidence cache: every tool result gets a stable ID; findings cite those IDs or they don't exist.
FastAPI orchestrator, concurrent dispatch with per-specialist crash isolation, JSON-persisted sessions, and a single model-call gateway so token accounting is complete and comparable.
228 tests, none requiring network or KiCad.

Results — and what I refuse to claim

6 KiCad boards, 12 reproducibly seeded defects, two independent runs per configuration (LLMs are nondeterministic; one run isn't a measurement).

	society	baseline (1× qwen3-max, all tools)
Seeded-defect recall (mean)	0.29	0.21
Cost per corpus	$0.147	$0.429
Prompt tokens	792K	312K
Findings surfaced	49–56	6–10
Hallucination rate	0.00	0.00

The society burns 2.5× MORE tokens and costs 2.9× LESS. That inversion is the whole thesis: five cheap specialists doing more work cost far less than one expensive generalist doing less. Routing by role beats routing by model size, and it reproduced to within 2% across runs.

I deliberately do not claim a detection win. The society led in both runs — by exactly one defect out of twelve. That's noise, and the README says so.

Challenges I ran into

Proving the society was better turned out to be harder than building it. My benchmark lied in my favour three separate times:

Agents left affected_nets empty, naming nets only in prose — so a correct catch scored as a miss. This wasn't just a scoring artifact: my report's headline visualization is built from those fields, so the centerpiece was silently broken.
My baseline was an accidental strawman. I'd named it moderator to inherit "all tools", but the registry scopes that agent down to three overview tools. It had been reviewing boards nearly blind.
The baseline was penalized on formatting. It found the defect but emitted title/description instead of claim/evidence, so every finding was discarded. The specialist prompts each carry a worked JSON example; the baseline's didn't. I was measuring prompt quality and calling it architecture.

All three pointed the same direction — none made the society look worse. That's what motivated reasoning looks like from the inside: you stop debugging when the number agrees with you.

What I learned

The reasoning layer is now ahead of the observability layer. Absolute recall is low for both configurations, and it's mostly the tools' fault: removing a decoupling capacitor creates an absence, and no KiCad tool reports that a part which should exist doesn't. A swapped SDA/SCL is electrically valid, so ERC stays silent. These models can reason about a missing bypass cap — they just have no instrument that reports it.

Also: evidence gating works. 0.00 hallucination rate across every run, both configurations. Make honesty structural and you stop arguing with the model about it.

One honest limitation: the negotiation engine — bounded rounds, evidence-cited rulings, fully unit-tested — never fired on real data. Across the whole corpus, no two specialists ever produced conflicting findings; their scopes are narrow enough that collisions are rare. The repo ships a clearly-labeled synthetic fixture to demonstrate the viewer and says so in the README, because the alternative is faking a debate and calling it a demo.

Alibaba Cloud

backend/app/qwen_client.py — all model calls via Model Studio, with per-agent token accounting
deploy/alibaba/oss_store.py — Alibaba Cloud OSS via the official oss2 SDK
deploy/Dockerfile — backend + KiCad container

What's next

More board classes, a KiCad plugin front-end, and — the interesting one — building the missing instruments: tools that detect absence, so the reviewers can finally see what isn't there.