InspectorMCP — agentic solar cell inspection

An AI agent that inspects solar panels: it scans every cell with OpenCV 5 on AWS Graviton, looks closer when unsure, then opens a repair ticket or asks a human.

Comment

Inspiration

A cracked solar cell quietly loses power and can turn into a hot spot. Electroluminescence (EL) photos make cracks visible as dark lines, but someone still has to look through thousands of cell images per site. Fully automatic checks are risky because borderline cells really are hard to call. We wanted an inspector that is fast on the easy cells, careful on the hard ones, and always lets a person make the final call when it is unsure.

What it does

InspectorMCP is an agent that inspects solar panels cell by cell:

  • Look — straightens the panel photo, splits it into cells, checks image quality.
  • Score — CellNet (a small CNN running in OpenCV 5's DNN module) rates each cell; classic OpenCV marks the cracks and dark areas.
  • Look closer when unsure — re-scores from six views, finds where the model sees the defect, zooms into that patch and measures the mark.
  • Act — passes the cell, opens a repair ticket with location and severity, or asks a human.

What the vision tools return decides the next tool call. Every step streams live in the web console, and unsure cells wait in an approval queue for a person. All 15 tools are also exposed over MCP, so any agent (Claude Code, Codex, Kiro…) can use them.

How we built it

  • OpenCV 5 (COOL) on AWS Graviton4 — EC2 c8g.large from the "Cloud Optimized OpenCV for AWS Graviton4" Marketplace AMI (OpenCV 5.1 + KleidiCV). All image work and the neural network run there.
  • Classic OpenCV: perspective warp, Otsu, black-hat morphology, connected components, minAreaRect, HoughLinesP, fitLine, CLAHE, occlusion maps.
  • CellNet: MobileNetV3-Small fine-tuned on the public ELPV dataset, exported to ONNX and run with cv2.dnn — 2 ms per cell.
  • MCP server (Python MCP SDK, Streamable HTTP) + a perception→decision→action agent with two brains: auditable rules, or rules + Gemini for uncertain cells.
  • FastAPI console with live Server-Sent-Events traces, approvals and tickets; front on Vercel.
  • AWS: EC2 Graviton4, Elastic IP, least-privilege IAM role, CloudWatch metrics and logs, Budgets alerts.

Results (655 held-out cells)

  • CellNet: 87.0% accuracy, 87% precision, AUC 0.91, 2.3 ms per cell (OpenCV 5 DNN).
  • Agent: decides 89% of cells by itself at 88.8% accuracy, sends 11% to a human (half of those really are defective).
  • Speed: profiling every OpenCV op made the full scan 3.7× faster (27 → 99 cells/s).
  • Cost: COOL on Graviton is 16% cheaper per cell than x86 (c7i.large).

Challenges we ran into

  • COOL wasn't automatically faster. Our first pipeline was 13% slower on COOL than the stock wheel, because it leaned on large median blurs, one of COOL's weak spots. Profiling single ops showed where COOL wins (CLAHE 1.48×, Otsu 1.22×, cubic resize 1.16×) and where it doesn't. Redesigning the hot path around that made the whole scan 3.7× faster.
  • Amazon Bedrock was blocked on our new AWS account ("Operation not allowed"), so the optional LLM brain uses the Gemini API instead.
  • Honest LLM results: Gemini cut human reviews on the hardest cells by 75%, but was only 59% accurate on what it decided. So rules stay the default and the human gate stays.
  • A unit test caught a real bug: large dark (inactive) areas were being masked out together with the cell's dark corners. Fixed with a convex-hull cell mask.

Accomplishments that we're proud of

  • A real agentic loop where vision output changes the next tool call, with full traces as evidence.
  • Every number in our report is reproducible from a script and a JSON file in the repo.
  • A fast, honest benchmark of COOL vs stock OpenCV vs x86, including the losses.

What we learned

Measure before optimizing. Matching the pipeline to the operations COOL accelerates mattered more than the library alone. And LLM agents still need a human gate on genuinely ambiguous cases.

What's next for InspectorMCP

  • Higher-resolution / tiled inference to catch tiny edge cracks (recall is 69% today).
  • Whole-module photos from real EL cameras.
  • Auth and persistence for the MCP endpoint, and Amazon Bedrock (Nova) as the LLM brain once our account is enabled.
Share this project:

Updates

Submission history