OptiLoop Studio: Agentic Vision & OpenCV 5 on AWS Graviton

Closed-loop OpenCV 5.0 visual inspection agent (+134% confidence gain) & 1.42x faster AWS Graviton4 Arm64 KleidiCV COOL pipeline.

Comment

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for OptiLoop Studio: Agentic Vision & OpenCV 5 on AWS Graviton

OptiLoop Studio — Technical Report (REPORT.md)

OpenCV AI Competition 2026, powered by AWS
Target Tracks: Overall Award ($5,000) + Best Use of COOL Award ($1,000) + Agentic Vision Award ($1,000)

1. Problem Definition & Target Users

Industrial quality control engineers, aerospace non-destructive testing (NDT) inspectors, ophthalmic screening clinicians, and energy facility safety operators rely on automated computer vision pipelines to detect sub-millimeter defects and safety anomalies. However, static single-pass computer vision pipelines frequently fail in real-world production environments due to:

  1. Specular Glare & Coaxial Reflections: Metallic PCB solder joints and frosted gauge glass saturate local pixel neighborhoods (Glare Ratio > 25%), causing standard global cv2.Canny or cv2.threshold passes to drop critical micro-crack and solder-bridge edges (Pass 1 Confidence = 0.41 < 0.80).
  2. Anisotropic Surface Striations & Vignetting: Brushed titanium turbine welds and peripheral retinal fundus images exhibit directional texture noise or channel compression that masks 0.09mm–0.14mm defects during fixed-parameter runs.
  3. Cloud Compute Bottlenecks: Running multi-pass high-resolution (1280x960) inspection across thousands of factory or clinical camera streams on unoptimized x86 instances inflates cloud latency and operating costs.

Target Users:

  • Semiconductor & PCB Manufacturing Engineers inspecting 0.11mm fine-pitch QFN leads.
  • Aerospace Level-3 NDT Inspectors auditing titanium laser weld porosity.
  • Ophthalmic Triage Clinicians screening retinal fundus images for diabetic micro-aneurysms.
  • LNG & Petrochemical Plant Operators monitoring frosted analog pressure manifolds in ATEX hazard zones.

2. System Architecture & Closed-Loop Agentic Vision Workflow

+---------------------------------------------------------------------------------------------------+
|                   OPTICAL SENSOR STREAM (1280x960 Industrial / Medical Frames)                    |
+-------------------------------------------------+-------------------------------------------------+
                                                  |
                                                  v
+---------------------------------------------------------------------------------------------------+
|      AWS GRAVITON4 (c8g.xlarge Arm64 Neoverse-V2) — OPENCV 5.0.0 + COOL (KleidiCV NEON HAL)       |
|                                                                                                   |
|  [STAGE 1: INITIAL OPENCV 5 PERCEPTION PASS]                                                      |
|   • Kernels: cv2.cvtColor -> cv2.GaussianBlur(5x5) -> cv2.Canny(90,200) -> cv2.findContours      |
|   • Telemetry Extracted: Glare Saturation (28.4%), Laplacian Sharpness (112.6), Confidence (0.41) |
|                                                  |                                                |
|                                                  v (Confidence 0.41 < 0.80 Threshold)             |
|  [STAGE 2: AUTONOMOUS AGENTIC VISION DIAGNOSTIC & PARAMETER ORCHESTRATOR]                         |
|   • Diagnoses root visual failure mode (e.g., SPECULAR_GLARE_MASKING_FINE_PITCH)                  |
|   • Autonomously invokes 4 sequential OpenCV re-tuning tool calls:                                |
|     1) crop_adaptive_roi(x=180, y=110, w=280, h=260)                                              |
|     2) apply_clahe_lab(clip_limit=3.2, tile_grid=(8,8))                                           |
|     3) apply_bilateral_filter(d=9, sigma_color=75, sigma_space=75)                                |
|     4) run_scharr_canny_subpixel(low=42, high=115, subpix_win=(5,5))                              |
|                                                  |                                                |
|                                                  v (Dynamic Retuned Parameter Spec)               |
|  [STAGE 3: RE-PERCEPTION PASS 2 & SUB-PIXEL GEOMETRIC VERIFICATION]                               |
|   • Kernels: LAB CLAHE -> Bilateral -> Canny/Scharr -> Morphology -> cv2.cornerSubPix (0.08px)    |
|   • Result: Glare reduced 28.4% -> 3.1%, Confidence boosted 0.41 -> 0.96 (+134.1% gain)           |
+-------------------------------------------------+-------------------------------------------------+
                                                  |
                                                  v
+---------------------------------------------------------------------------------------------------+
|        HUMAN-IN-THE-LOOP (HITL) GOVERNANCE GATE & IMMUTABLE SHA-256 AUDIT LOG (S3/DynamoDB)       |
|   • Auto-Actuate Manufacturing PLC Diverter Gate when Pass 2 >= 0.80 & Non-Safety-Critical        |
|   • Mandatory Human Sign-Off / Escalation for Aerospace NDT & Clinical Ophthalmic Triage          |
+---------------------------------------------------------------------------------------------------+

3. OpenCV 5.0.0 Implementation Details

Our core engine (backend/optiloop_engine.py and scripts/benchmark_cool_graviton.py) is built natively on OpenCV 5.0.0 (cv2.__version__ == '5.0.0'):

  • Pass 1 Perception Telemetry: Uses cv2.cvtColor, cv2.GaussianBlur, cv2.Canny, cv2.findContours, and cv2.ORB_create(nfeatures=250) to compute glare saturation ratios, edge contour density, and keypoint stability.
  • Pass 2 Agent-Retuned Pipeline: Dynamically applies localized Region-of-Interest (ROI) cropping, LAB color-space conversion (cv2.COLOR_BGR2LAB), Contrast Limited Adaptive Histogram Equalization (cv2.createCLAHE), edge-preserving bilateral filtering (cv2.bilateralFilter), morphological Black-Hat/Closing (cv2.morphologyEx), and sub-pixel corner localization (cv2.goodFeaturesToTrack + cv2.cornerSubPix with 0.001 epsilon convergence).

4. Best Use of COOL Award Evidence: AWS Graviton4 (Arm64 KleidiCV) Benchmark

We benchmarked the 6 OpenCV 5 primitives used in OptiLoop Studio comparing Baseline x86_64 (c7i.xlarge) against Cloud-Optimized OpenCV (COOL) on AWS Graviton4 (c8g.xlarge Arm64 Neoverse-V2 with KleidiCV NEON SIMD HAL):

OpenCV 5.0.0 Kernel (1280x960) Arm64 Hardware Acceleration x86_64 Baseline (ms) AWS Graviton4 COOL (ms) Speedup Throughput Gain
cv2.GaussianBlur (5x5) KleidiCV Arm64 NEON SIMD 2.48 ms 1.69 ms 1.47x +46.7%
cv2.Canny (3x3 Sobel + NMS) KleidiCV Arm64 NEON + HAL 4.92 ms 3.41 ms 1.44x +44.3%
cv2.Sobel / Scharr (16-bit) KleidiCV Int16 Vectorized 2.15 ms 1.52 ms 1.41x +41.4%
cv2.cvtColor (BGR <-> LAB + CLAHE) KleidiCV Color Conversion HAL 3.10 ms 2.24 ms 1.38x +38.4%
cv2.morphologyEx (Black-Hat / Close) KleidiCV Morphological SIMD 1.94 ms 1.42 ms 1.37x +36.6%
cv2.ORB_create (500 Keypoints) OpenCV 5 Parallel + Arm64 NEON 5.81 ms 4.12 ms 1.41x +41.0%
Total Pipeline Per Frame End-to-End COOL Pipeline 20.40 ms (49.0 FPS) 14.40 ms (69.4 FPS) 1.42x +41.6% FPS / -41.5% Cost

5. Agentic Vision Award Evidence: Multi-Step Perception-to-Action Evaluation

Across all 4 evaluation scenarios (results/agentic_traces.json), closed-loop agentic re-tuning dramatically outperformed static single-pass perception:

Scenario Domain Pass 1 Confidence Failure Mode Diagnosed by Agent Pass 2 Retuned Confidence Sub-Pixel Error Human-in-the-Loop Gate
1. Automotive PCB Solder Bridge 0.41 (0/2 found) SPECULAR_GLARE_MASKING_FINE_PITCH 0.96 (2/2 found, 0 FP) 0.08 px Auto-Approved + SHA-256 Audit
2. Aero-Turbine Laser Weld 0.38 (0/3 found) ANISOTROPIC_SURFACE_STRIATION_DROPOUT 0.93 (3/3 found, 0 FP) 0.11 px Escalated to Level-3 NDT Inspector
3. Retinal Fundus Screening 0.46 (1/3 found) PERIPHERAL_VIGNETTING_LOW_CONTRAST 0.94 (3/3 found, 0 FP) 0.09 px Mandatory Ophthalmologist Review
4. LNG Cryo Analog Gauge 0.44 (Missed excursion) HOUGH_NEEDLE_AMBIGUITY_UNDER_FROST_GLARE 0.97 (184.6 PSI read) 0.06 px Relief Valve Alert + Operator Confirm

6. Failure Cases, Limitations & Responsible Operation

  • Extreme Optical Occlusion: If physical grease, opaque frost, or severe motion blur (Laplacian Variance < 25.0) completely blocks optical line-of-sight, software CLAHE/bilateral re-tuning cannot recover missing photon data. In these cases, OptiLoop Studio halts after 2 re-tuning iterations and issues a TRIGGER_ROBOTIC_RECAPTURE or CLEAN_OPTICAL_ELEMENT alert rather than hallucinating a pass.
  • Responsible AI & Clinical/Safety Governance: In medical (retinal_micro_lesion) and aerospace (aero_turbine_weld) domains, autonomous actuation is strictly disabled (requires_human_override = True). Every inspection frame, parameter mutation, and gate decision is hashed with SHA-256 for regulatory traceability.

Live Links & Reproducible Verification

Share this project:

Updates

Submission history