Agentic Visual Inspector

The problem

Factory QA lines miss subtle defects because inspectors fatigue, and they raise false alarms because a detector cannot weigh context. An isolated blemish and the third identical blemish in an hour are not the same event, but a per-frame detector reports them identically. Detection is a commodity; the decision is where the safety and the money are.

What it does

Two stages, and the second is the point:

• See — OpenCV 5's DNN engine runs a YOLOv10 ONNX detector over a frame, on CPU, with no cloud on the critical path. Verified: 5 detections (a bus and four people) in ~110–135 ms on a laptop CPU.
• Reason — an agent receives the detection plus the recent history on that line and investigates: is it critical, which prior defect does it resemble, what is the trend, what should happen next. The output is a structured decision with a reasoning trace a supervisor can read and challenge — not a bare box.

The agentic part

A single prompt is not agency. The reasoner is a bounded tool loop (at most two calls): it can fetch more of the line's history when the window looks thin, then returns strict JSON. Retrieval ranks class match first and recency second — the two signals a human supervisor actually uses when they say "this looks like the one from an hour ago". The system prompt forbids inventing history, and a test enforces it: any cited prior must be one the agent was actually given.

AWS

The reasoning inference is the AWS component. The pipeline ships two identical-contract backends — a local OpenAI-compatible one used for development, and Amazon Bedrock (Converse API) — selected by one environment variable. The loop, prompt and output contract do not change; only the transport does.

What is verified

• OpenCV 5 + ONNX detection on CPU: 5 detections in ~110–135 ms, no GPU.
• The agent investigated the top detection, judged it HIGH, matched a prior, and recommended a halt, with a readable trace.
• A working web endpoint (POST /api/inspect) returns detections, the decision and the annotated frame; driven end to end in a browser with zero console errors.
• A pipeline test suite passes 3/3, including the guard that the agent never cites history it was not given — shown to be non-vacuous.

Honest limits

The Bedrock path is implemented and wired but has not been run against a live AWS account, because none was available during development. The detector is a general COCO model standing in for a plant-floor defect model; only the weights change. There is no labelled defect corpus yet, so the numbers above are functional and latency measurements, not precision/recall. Human control stays in the loop: the system recommends and escalates, it does not actuate machinery.

Built With

• agentic-ai
• amazon-web-services
• computer-vision
• machine-learning
• opencv
• python