Agentic Visual Inspection using Guardian CV

An OpenCV 5-powered inspection agent that detects visual anomalies, investigates suspicious regions through adaptive analysis, verifies findings with evidence-driven decisions and AWS processing.

Comment

Inspiration

Traditional computer-vision systems can detect visual differences, but identifying a difference is only the beginning of an investigation. Lighting variations, shadows, camera displacement, and image noise can create false alarms, while meaningful changes may require closer examination.

We asked ourselves: What if a visual inspection system could investigate what it sees, decide what to examine next, and verify its findings before reaching a conclusion?

This question inspired Guardian CV, an agentic visual inspection platform that combines computer vision with evidence-driven decision-making to make visual inspection more adaptive, transparent, and useful.

What It Does

Guardian CV transforms visual comparison into a structured investigation workflow.

Users provide reference and current images, and the system analyzes the visual evidence to identify potential anomalies. Rather than treating every difference as a confirmed defect, Guardian CV is designed to investigate suspicious regions, select appropriate follow-up actions, and reassess its findings.

Its core capabilities include:

  • Visual comparison: Analyze reference and current images using OpenCV 5-based processing.
  • Anomaly localization: Generate difference maps and highlight regions that may require attention.
  • Adaptive investigation: Select follow-up analysis actions based on the evidence collected during inspection.
  • Evidence-based verification: Reassess findings after additional analysis and distinguish supported findings from uncertain outcomes.
  • Traceable decisions: Present the investigation steps, measured results, and reasons behind the final assessment.
  • Human oversight: Identify ambiguous cases that need further evidence or human review.
  • Cloud-powered processing: Use AWS for a meaningful part of the inspection workflow.

The goal is not simply to detect what changed, but to investigate why a change may matter and determine whether the available evidence supports a conclusion.

How We Built It

We designed Guardian CV around a modular architecture that separates visual perception, investigation planning, tool execution, verification, and reporting.

1. Computer vision with OpenCV 5

The vision pipeline provides the foundation for image processing, reference-to-current comparison, difference-map generation, and region-level analysis. The selected techniques are intended to help identify potential anomalies while accounting for factors that can affect comparison quality.

2. Evidence-driven agentic workflow

The agent follows an Observe → Assess → Decide → Act → Reinspect → Verify → Conclude cycle. Findings from the current analysis influence the next action, allowing the system to investigate uncertain or suspicious regions instead of relying solely on a fixed sequence of operations.

3. Controlled investigation tools

The system is structured around individual analysis operations, allowing follow-up actions to be executed, evaluated, and recorded. This makes the investigation process more transparent and easier to test.

4. AWS integration

AWS supports cloud execution within the processing workflow. The architecture is designed to connect inspection requests, visual analysis, investigation results, and generated evidence.

5. Evidence and reporting

Inspection results are organized around visual findings, analysis steps, uncertainty, and the final outcome. This approach helps users understand how the system reached its assessment rather than relying on an unexplained result.

Challenges We Ran Into

One of our biggest challenges was distinguishing meaningful visual anomalies from harmless differences. Changes in illumination, camera positioning, shadows, blur, and image quality can all affect comparison results. A reliable inspection workflow must recognize these limitations instead of presenting every highlighted region as a defect.

A second challenge was making the agentic component meaningful. An agent that merely describes an image is not truly conducting an adaptive investigation. The difficult part is connecting measured visual evidence to a suitable next action, executing that action, and evaluating whether the additional evidence changes the assessment.

We also had to consider the consistency of local and cloud execution, reproducibility, processing failures, and how to communicate uncertainty without overwhelming users.

These challenges shaped our focus on observable decisions, evidence-backed findings, explicit verification criteria, and human review when the available evidence is insufficient.

Accomplishments That We're Proud Of

We are proud of the direction Guardian CV takes: bringing computer vision and agentic decision-making together in a focused visual inspection workflow.

Our key goals and engineering contributions are:

  • Turning visual comparison into a multi-step investigation rather than stopping at an initial difference map.
  • Designing a workflow in which visual evidence can influence the next analysis action.
  • Making the investigation process understandable through traceable actions and evidence.
  • Treating uncertainty and human review as important parts of the system rather than hiding inconclusive results.
  • Connecting the inspection workflow with AWS-powered processing.
  • Building around a modular architecture that supports testing, reproducibility, and future extensions.

Most importantly, we are working toward a system whose decisions can be examined and evaluated, not merely accepted because an AI produced them.

What We Learned

Building Guardian CV reinforced that useful computer vision involves much more than selecting an algorithm. Input quality, alignment, noise, uncertainty, evaluation, and failure handling all influence the usefulness of an inspection result.

We also learned that agentic behavior must be demonstrated through actual actions and changing decisions. A meaningful agent uses the output of one operation to determine what to do next, evaluates the new evidence, and knows when to stop or request human input.

Finally, we learned the importance of balancing ambition with engineering discipline: measurable results, reproducible tests, clear limitations, and a reliable end-to-end demonstration are more valuable than a long list of unverified features.

What's Next for Agentic Visual Inspection Using Guardian CV?

Our next step is to strengthen Guardian CV as a practical, evidence-driven inspection platform.

We aim to improve robustness under lighting changes and camera displacement, extend inspection to video and repeated observations, and evaluate the system across more diverse visual conditions. We also plan to strengthen benchmark comparisons against conventional one-pass image-difference methods to measure when adaptive investigation genuinely improves outcomes.

Further development will focus on improving the accuracy of region localization, making uncertainty estimates more informative, expanding reproducible evaluation datasets, and refining the evidence report and human-review workflow.

Over time, Guardian CV could support specialized inspection workflows in manufacturing, maintenance, and quality assurance, with domain-specific validation before use in consequential settings.

Our vision is simple: move from computer vision that merely detects differences to visual systems that investigate evidence, verify findings, and communicate what they can—and cannot—conclude.

Built With

  • agentic-ai
  • amazon-web-services
  • anomaly-detection
  • artificial-intelligence
  • cloud-computing
  • computer-vision
  • explainable-ai
  • image-analysis
  • image-comparison
  • image-processing
  • image-segmentation
  • opencv
  • python
  • visual-inspection
  • web
Share this project:

Updates

Submission history