ListingLens

A quality gate for product photos: an OpenCV 5 agent fixes only what it can prove it fixed, rolls back what it can't, and tells the seller exactly what to reshoot.

Comment

Inspiration

Small online sellers shoot product photos on a phone, in a kitchen or a warehouse. Marketplaces want a sharp, neutral, centred product on a plain light background, and a seller usually learns a photo failed only after the listing is rejected or quietly ignored by buyers. Retouching every photo by hand is slow, and "AI enhancement" that repaints the product is the wrong answer: a product photo has to stay true to the item in the box.

What it does

ListingLens is a quality gate that sits between the seller's camera and the store listing. You drop the photos for one listing (or click a sample). For each photo the agent:

  1. Perceives with OpenCV 5: product mask, framing, edge contact, background clutter and colour cast, exposure, blur (Crete-Roffet index + Laplacian variance), noise, perceptual hash for duplicates.
  2. Decides one next action from the actions that are legal for that evidence.
  3. Acts with a deterministic OpenCV tool: recrop, white balance, exposure, GrabCut background replacement, non-local-means denoise, mild sharpening.
  4. Verifies by measuring again. If the action did not fix its target, created a new problem, shifted the product's colour, or GrabCut disagreed with the mask, it is reverted and that tool is ruled out.

It ends with one of four answers, each with evidence: ready to publish, fixed automatically (with the corrected photo), reshoot (with a concrete tip such as "move closer"), or a person should decide (with Approve / Reject buttons). Scene and lifestyle photos are recognised and never cut out.

How we built it

  • OpenCV 5.0 for all perception and all tools; NumPy; Python 3.12.
  • AWS Lambda on arm64 (Graviton) with a public Function URL, OpenCV 5 and NumPy as Lambda layers, CloudWatch Logs (one JSON line per request, no pixels), an hourly invocation cap that switches the function off through CloudWatch, SNS and a kill-switch Lambda, and an AWS Budgets alert. Everything serves inside the AWS free tier.
  • COOL: the same workload ran on EC2 c8g.large (Graviton4) from the COOL 3.1 Marketplace AMI, side by side with the stock opencv-python-headless wheel on the same instance, two runs of three alternating rounds each, with a per-stage profile.
  • Evaluation set built from Amazon Berkeley Objects (CC BY 4.0): 252 clean catalog photos x 13 controlled defects = 3,276 labelled photos, split into dev (tuning) and held-out test; plus 106 real catalog photos labelled by hand.
  • An optional Amazon Bedrock (Nova) planner is implemented and unit-tested: it must choose from the legal action list and falls back to rules otherwise. The public demo uses the rule planner to stay in the free tier.
  • Built by one person with an AI coding assistant (Claude Code) for implementation; the agent loop itself runs without it.

Results

  • 89.6 % correct decisions on 1,768 held-out test photos (87.5 % with the strict answer sets written down before any result). The same rules and tools without the re-measure step: 85.2 %, and 255 of its 1,032 "fixed" photos still fail the checks.
  • On the 371 test photos where the agent rolled back one of its own actions, it reached the right decision 94.6 % of the time (open loop on the same photos: 73.9 %).
  • 106 hand-labelled real catalog photos: 27 of 33 problem photos stopped (open loop: 11 of 33); 18 of 23 "accept" verdicts correct. The cost: about four in ten good photos still go to a person.
  • 100 % on strong blur, low resolution, dark photos and cluttered backgrounds; weakest on overexposed white products (33 %).
  • COOL on Graviton4: box filter 2.87x, Otsu 1.94x, CLAHE 1.40x faster; whole agent 1.03x. A per-stage profile shows why: perception is about 40 % of the time and about 80 % of it is NumPy, which COOL cannot reach. adaptiveThreshold 0.77x and Canny 0.90x were slower. Decisions identical in all twelve runs.

Challenges

  • A first version "fixed" room scenes by cutting furniture out of context. A scene check on the GrabCut foreground now refuses those.
  • Professional catalog photos looked "noisy" because fabric and print texture fooled the noise estimator; measuring noise on flat regions only fixed it.
  • White products on white backgrounds look like blown highlights. That is still our weakest case (33 % on overexposed test photos) and it goes to a person instead of forcing a reshoot.

What we learned

Measuring after every action is what makes an agent trustworthy: most of our quality gains came from better verification, not from better fixes.

What's next

Per-marketplace policies (Amazon, Etsy, Rozetka), moving the perception math from NumPy into OpenCV primitives so COOL's fastest kernels reach it, and evaluation on real phone photos labelled by sellers.

Try it

Photos in the samples and evaluation: Amazon Berkeley Objects, CC BY 4.0.

Built With

Share this project:

Updates

Submission history