ListingLens Agent

An OpenCV 5 catalog-photo QA agent that measures, acts, re-measures, and knows when to ask a human.

Comment

Inspiration

Product and catalog photography has a practical QA gap: an image can be sharp and well exposed yet still fail because the object is too small, off-center, or poorly framed. A fixed classifier can describe that problem, but ListingLens is designed to act on it safely.

What it does

ListingLens is an OpenCV 5 agentic vision workflow built around a bounded perception → decision → action → re-perception loop.

OpenCV 5 measures:

  • sharpness (variance of Laplacian);
  • mean luminance and clipping;
  • border/background variation;
  • foreground bounding box;
  • object occupancy;
  • center offset.

A deterministic policy then chooses:

  • ACCEPT — image already satisfies policy;
  • ACCEPT_AFTER_FIX — a recoverable composition problem triggers an automatic crop, then OpenCV measures the corrected image again before acceptance;
  • HUMAN_REVIEW — bounded correction was attempted but still does not pass;
  • HUMAN_RESHOOT — blur, severe exposure failure, missing/uncertain subject, or unsafe correction is handed to a human.

Every run keeps the full evidence trace: measurements, decision, action, optional corrected artifact, re-measurement, and final decision.

Why it is agentic vision

OpenCV output changes the next action rather than serving as decorative context.

In the deterministic margin/off-center case, initial perception measures about 0.085 occupancy and 0.266 center offset. Those values trigger an auto-crop. The corrected frame is measured again at about 0.651 occupancy and 0.0 center offset, and the final decision becomes ACCEPT_AFTER_FIX.

Blur, darkness, and overexposure deliberately route to HUMAN_RESHOOT instead of silently modifying product pixels.

How I built it

The core is Python with OpenCV 5.0.0 and NumPy. Perception and policy are separated so every action is inspectable and reproducible.

The AWS path is designed as:

S3 incoming/
  → AWS Lambda container (arm64)
  → OpenCV 5 perception
  → deterministic policy
  → optional bounded crop
  → OpenCV 5 re-perception
  → result.json + optional corrected.png back to S3

The repository includes AWS SAM infrastructure, scoped S3 read/write permissions, an S3 event trigger, local fake-S3 integration tests, CI, a GitHub Pages evidence demo, and a Remotion judge-video source.

The current repository proves the SAM template validates and the arm64 Lambda container builds locally. I do not claim a live AWS deployment until a real cloud receipt exists.

Evaluation

Five deterministic synthetic cases cover both success and failure handling:

  1. good frame → ACCEPT
  2. excess margin + off-center → AUTO_CROP → re-perception → ACCEPT_AFTER_FIX
  3. blur → HUMAN_RESHOOT
  4. dark exposure → HUMAN_RESHOOT
  5. overexposure → HUMAN_RESHOOT

Current verified evidence:

  • 7/7 Python tests pass;
  • 5/5 synthetic cases match expected decisions;
  • public evidence demo returns HTTP 200;
  • committed evidence reports OpenCV 5.0.0;
  • Remotion renders an 80-second 1920×1080 H.264 judge cut.

Challenges

The main challenge was keeping autonomy useful without making it reckless. Automatic correction is limited to a narrow composition fault. Blur and severe exposure problems are escalated because aggressive pixel rewriting would make catalog QA less trustworthy.

A second challenge was observability: the system therefore exposes the measurements that caused each action instead of returning only a final label.

Accomplishments

  • OpenCV 5 materially changes later tool/action behavior.
  • The system performs an autonomous repair followed by re-perception.
  • Failure modes remain bounded and human-controlled.
  • The evidence chain is inspectable.
  • Tests, CI, public evidence, AWS packaging, and a reproducible judge-video source are all in place.

What I learned

Agentic vision is most convincing when autonomy is narrow, observable, and reversible. The useful result is not just “fix the image”; it is “fix it for a measurable reason, measure again, and refuse unsafe automation.”

What's next

Record the live AWS deployment/e2e receipt, update the cloud-delivery scene in the judge video, and publish the final unlisted judge cut.

Built With

Share this project:

Updates

Submission history