REFLEX

Live AWS v1 · real VisA candle · original / heatmap / overlay. Amazon Science, CC BY 4.0. Curated demo, not an accuracy estimate.
Live AWS v1 · real VisA cashew · evidence and decision. VisA / Amazon Science, CC BY 4.0; heatmap and overlay are derived.
Measured benchmarks and known failures. VisA results, generated procedure tests and software checks remain distinct populations.
REFLEX: OpenCV perception → bounded decision → detail/review → evidence. V2 detail is local; AWS demo runs v1.

Inspiration

A low anomaly score does not guarantee a defect-free product. A blurred capture, a tiny defect or an unfamiliar texture can make a single inspection inconclusive. We wanted an inspector to see the original photograph, the model's evidence and the reason for the next action in one workspace.

REFLEX explores a practical question: when should a vision system stop, inspect a detail, or ask a person for better evidence? Its intended users are quality-control teams, machine-vision integrators and engineering reviewers. It is an inspection prototype, with human control over consequential decisions.

What it does

An inspector chooses a fitted product profile and uploads a photograph or selects a licensed real VisA example. REFLEX validates the input, compares local visual features with a memory of normal products, and returns the original image, anomaly heatmap, overlay, score, thresholds and tool trace. A matching clean photograph is not required for each inspection.

The local v2 candidate adds a bounded second look. Image-quality flags can request a retake. A middle-band score can trigger one native-source crop, chosen from the first anomaly map and evaluated against a separately calibrated detail memory. Inconclusive evidence stays NEEDS_REVIEW. A crop is a closer view of the same photograph, not a newly acquired camera image.

An operator can append a review without overwriting the machine finding, and export JSON or a printable HTML evidence report. A separate local Safety Station prototype checks the observed order of steps on a calibrated colored-parts teaching panel using actual decoded video frames. Its supplied clips are explicitly generated software tests.

Deployment boundary: the public AWS endpoint runs the earlier v1 real-image detector for candle and cashew profiles. The adaptive v2 candidate and Safety Station are local capabilities; they are not presented as deployed AWS features.

How we built it

The interface uses HTML, CSS and JavaScript. Python, FastAPI and Uvicorn serve the API. OpenCV 5 performs image decoding and normalization, ONNX inference, spatial map processing, contour extraction and evidence rendering. Fixed ImageNet-pretrained ResNet-50 features are compared with normal-only patch memories; industrial defect labels are reserved for evaluation. This is not a fine-tuned defect classifier or the official PatchCore implementation.

The bounded inspection policy is deterministic: image quality, scores and spatial evidence determine later actions. It is not an LLM explaining a fixed result. The local procedure prototype separately uses OpenCV video decoding, perspective transforms, color/contour observations and temporal recipe checks. An optional pair-alignment utility uses ECC; it is not silently inserted into the anomaly pipeline.

On AWS, ECS Express provides HTTPS ingress and Fargate runs the Python/OpenCV container in ap-southeast-2. ECR stores the image, CodeBuild tests and builds it from a private S3 source bundle, and CloudWatch receives structured metadata logs. Inspection records use task-local SQLite and PNG files; runtime images are not stored in S3. Task replacement can remove these records before their normal expiry.

The architecture and editable eight-slide presentation are included in the supporting archive. The repository contains pinned dependencies, model-download hash checks, reproduction instructions and measured benchmark tables.

What we measured

We evaluated 860 distinct real VisA photographs across candle, cashew, PCB1, capsules and chewinggum. The populations remain separate: 350 exposed replay images, 255 diagnostic-development images and a custom 255-image final partition. Eleven methods/arms across eight groups produce 88 result rows; repeated predictions are not additional independent images.

The fixed primary v2 candidate detected 155/200 replay defects, compared with 112/200 for v1. Four replay defects remained under review. On the new-category final partition, v2 detected 105/150 defects, with 42 automatic false clears and 3 unresolved defect reviews. It also produced 14 normal false alerts and 8 normal reviews among 105 normal cases.

The limitations matter as much as the improvement:

Final category-macro image AUROC fell from 0.8520 to 0.8410. Capsules remained a major failure: 34 of 50 final defects were automatically cleared.
The upstream PatchCore reference ranked higher in all eight measured groups under a different backbone, memory and preprocessing configuration. We do not claim a controlled architecture-only comparison or state-of-the-art performance.
A detail crop ran on 46/860 images. Matched policy controls show that the review boundary, rather than crop selection alone, accounts for the reduction in automatic false clears. General superiority over random crops is not established.
Dedicated uncached Fargate handler measurements used 300 requests per version on 1 vCPU / 2 GiB. V1 p95 was 1.070 s and v2 p95 1.256 s; peak process memory also increased. These are handler measurements, not network latency or production capacity.
Nine dependent image transformations produced 5,445 robustness cases. They are stress tests derived from existing photographs, not independent real captures.
The separate procedure software benchmark contains 18 generated test clips and 54 policy runs. All policies tie at 15/18 final statuses and 5/7 timely typed errors, with no observed agent accuracy gain. Physical procedure evaluation is NOT_RUN.

The public evidence ledger retains outcomes and failures. Historical custom AU-PRO values are marked invalid; the corrected evaluator and its remaining publisher-parity limitation are documented. Software test counts are reported separately from model performance.

Challenges we ran into

Small defects can disappear at full-frame input resolution, while arbitrary crops can change the normal feature distribution. This led us to separate full-image and detail calibration. Another challenge was preventing review cases from being counted as successful detections: we now report every original defect, including unresolved reviews.

The experiments also challenged our product intuition. More adaptive processing did not automatically improve final ranking or cloud efficiency. We held the v2 release instead of promoting a candidate solely because its replay score improved. Public untrusted video decoding, persistent shared storage and organizational access controls also require more work before broader deployment.

Accomplishments we're proud of

We have a working AWS vision application that analyzes real photographs and exposes inspectable evidence, an implemented bounded second-look candidate, and a reproducible study that keeps negative results visible. The 25 October audit recorded 138 passing Python tests, 14 frontend tests, 5 markup checks and 20 separate API assertions. Those checks establish their tested software behavior, not factory readiness.

What we learned

Review is a first-class outcome. Ranking metrics, thresholded detection, missed defects and operator workload answer different questions. A visually attractive heatmap is not proof that the decision is correct, and an extra tool call needs a measured benefit to justify its cost.

What's next for REFLEX

Acquire authorized fresh captures separated by physical item and lot; compare single-pass, review-only, chosen-crop and random-crop policies under matched budgets; measure registration's downstream effect; and validate real procedure recordings with independent labels. A practitioner pilot should measure misses, false alerts, interventions and verified inspections per reviewer-hour before making ROI or safety claims.

Try it and review the evidence

Use only licensed, non-private demo images on the public endpoint. It has no user login or tenant isolation. A PASS is not a safety certificate, and reviewer feedback does not automatically train the model. We used AI coding assistance during development; the implemented inspection policy, not the coding assistant, is the adaptive vision component described above.

Real example photographs: VisA, Amazon Science, CC BY 4.0, from Zou et al., SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation, ECCV 2022. Heatmaps and overlays are derived outputs. Curated screenshots demonstrate the interface and individual runs; the benchmark tables establish the measured dataset results.