ListingLens

A quality gate for product photos: an OpenCV 5 agent fixes only what it can prove it fixed, rolls back what it can't, and tells the seller exactly what to reshoot.

Comment

The one idea: ListingLens never trusts its own fix. After every OpenCV action it measures the photo again and rolls the action back if the numbers say it did not help. On the 371 held-out photos where it rolled back one of its own actions, it still reached the right decision 94.6 % of the time. The same pipeline without the re-measure step got 73.9 % on those photos.

Judge's 2-minute path

  1. Open the live demo: https://je7md23hxr42uzjol5luyteg2q0axshn.lambda-url.us-east-1.on.aws/ (the first request after a quiet period takes a few seconds while Lambda starts).
  2. Click three sample photos and compare with what you should see:
    • colour cast -> "Fixed automatically", before/after, the trace shows white balance kept after re-measuring;
    • motion blur -> "Reshoot" with a concrete tip, because blur cannot be fixed honestly;
    • lifestyle -> "A person should decide": the agent tries to replace the background, measures that the product runs past the frame (a scene, not a packshot), rolls its own edit back and hands over. Timing, measured from Kyiv on a warm function: the lifestyle sample takes about 5 s (GrabCut runs and is then rolled back); the other samples answer in 1-3 s.
  3. Or upload your own product photo. Nothing is stored. The gate is strict on purpose: a casual phone snapshot on a table usually goes to a person. "How to shoot a photo the gate accepts" below lists what it measures.
  4. Evidence, one click each: technical report (PDF) · full agent trace · evaluation summary · OpenCV build info of the live function · source code (sha256 starts with 3e97b949, same file as the attached archive).
  5. Code on GitHub (public, tag v1.1-judging, same content as the archive): https://github.com/DmitriyTopka/listinglens

In plain numbers

  • Out of 100 held-out test photos with a known defect or none, about 90 get the right decision (1,584 of 1,768; 95 % CI 88-91 %). Without the re-measure step: about 85. On the same photos the agent is right where the open loop is wrong 97 times, the reverse happens 20 times (McNemar p < 0.000001).
  • On 106 real catalog photos labelled by hand before the run: of 33 photos with a real problem, 27 are stopped before publishing (82 %, 95 % CI 66-91 %). An open-loop pipeline stops 11 of 33. Every photo the open loop stopped, the agent stopped too (16 vs 0 discordant, exact McNemar p = 0.00003).
  • The price we pay for that: about four in ten good photos still go to a person instead of straight to the shop (31 of 73, 95 % CI 32-54 %). We chose a false alarm over a bad listing. Of the 73 good real photos, 30 pass automatically (18 accepted as they are, 12 auto-fixed), 31 go to a person and 12 are sent to reshoot.
  • About 1 second per photo on AWS Lambda arm64 (Graviton). Even at the x86 list price ($0.0000166667 per GB-second, 2 GB, about 1.2 s billed) that is under 5 cents per 1,000 photos, and about 160,000 photos a month fit inside the always-free tier.

How to shoot a photo the gate accepts (a phone is fine)

No studio needed, but the gate measures these things, so this is what to aim for:

  • Send the original file. The short side must be at least 1,000 px. Messengers shrink photos (Telegram gave us 960 px), so use AirDrop, a cable or email "as file".
  • A plain sheet behind and under the product, filling the whole frame: printer paper, a sheet of whatman or a matte white board. The gate reads the outer 4 % of the frame, and if more than 2 % of it is edges (wood grain, fabric, a table edge, a hand) it calls the background busy.
  • Even light over that sheet. Along the frame edge, lightness may vary by about 12 % at most (standard deviation 9 on a 0-255 scale). A window from one side is the usual reason it fails: daylight from the side darkens the far edge. Overcast light from above, or two lamps on opposite sides, works better.
  • The whole product with a margin on every side. Touching an edge sends the photo to a person, running along a fifth of an edge means reshoot. Too small or off-centre is fine: the agent tries a recrop and re-checks it.
  • Sharp. Rest the phone on something and tap the product to focus. Mild softness is sharpened once and re-checked; real blur means reshoot.
  • Don't blow out white products. If more than 30 % of the product is pure white, a person decides. Slightly darker is fine: for exposure, a colour tint or an off-white background the agent applies a fix and keeps it only if the re-measure passes.

These rules are written from the thresholds in listinglens/policy.py. We have not yet confirmed them with a fresh set of phone photos shot this way.

Why measurements and rules, not an LLM, make the call

A seller needs to know why a photo was rejected, and the same photo must get the same answer tomorrow. Every decision here is a measured number against a written threshold, and the verify step lives in code, so even a wrong plan cannot publish a worse photo. An Amazon Nova planner is implemented (Bedrock Converse tool use): it may only choose from the actions that are legal for the evidence and falls back to rules otherwise. The public demo uses the rule planner so it stays free and deterministic.

COOL, stated precisely

COOL ran our batch catalog check on EC2 c8g.large (Graviton4) from the COOL 3.1 Marketplace AMI: two full benchmark runs on separate instances, each with 3 COOL and 3 stock OpenCV 5 batch runs of 260 photos alternating on the same machine. Decisions were identical between COOL and stock in all 12 runs. OpenCV kernels got faster (box filter 2.87x, Otsu 1.94x, CLAHE 1.40x; adaptiveThreshold 0.77x slower), the whole agent only 1.03x, because about 80 % of our perception time is NumPy, not OpenCV. The live demo on Lambda runs stock opencv-python-headless 5.0, since COOL ships as an EC2 image, not as a Lambda layer.

Where the agent's limits are (measured, not guessed)

We took 40 clean held-out photos and pushed one defect at a time from none to strong, without touching the deployed thresholds. Blur: up to sigma 0.8 px most photos pass, from 1.6 px almost all go to reshoot. Darkness and colour cast: fixed automatically from the first visible step. Noise: below sigma 8 nobody would see it and the agent accepts; above it, it denoises or asks for a reshoot. Overexposure is our measured blind spot: a white product on white gets brighter and brighter, and even at full strength 40 % of photos still pass as "accept"; the rest are fixed or sent to a person. We show this instead of hiding it, and it is the first thing we would work on next.

Real phone photos. We shot 24 everyday products on a phone, labelled them before the run (8 bad, 16 fine; labelled by the author's assistant, not an independent person), committed the hashes and ran the agent once. All 8 bad photos were stopped, and so were all 16 good ones (0 of 16 passed, 95 % CI 0-19 %). The photos had passed through Telegram at 960 px, under the 1,000 px rule. A diagnostic re-run without that rule showed the second cause: wood grain, and the light falloff from a window across a white sheet, both count as a busy background. Six of the white-sheet scenes re-sent at 1512x2016 confirm it: the resolution rule no longer fires, the window light still sends them to a person. We did not change any threshold after seeing these photos. Full table and diagnosis: eval/phone24/RESULTS.md in the source archive.

Inspiration

Small online sellers shoot product photos on a phone, in a kitchen or a warehouse. Marketplaces want a sharp, neutral, centred product on a plain light background, and a seller usually learns a photo failed only after the listing is rejected or quietly ignored by buyers. Retouching every photo by hand is slow, and "AI enhancement" that repaints the product is the wrong answer: a product photo has to stay true to the item in the box.

What it does

ListingLens is a quality gate that sits between the seller's camera and the store listing. You drop the photos for one listing (or click a sample). For each photo the agent:

  1. Perceives with OpenCV 5: product mask, framing, edge contact, background clutter and colour cast, exposure, blur (Crete-Roffet index + Laplacian variance), noise, perceptual hash for duplicates.
  2. Decides one next action from the actions that are legal for that evidence.
  3. Acts with a deterministic OpenCV tool: recrop, white balance, exposure, GrabCut background replacement, non-local-means denoise, mild sharpening.
  4. Verifies by measuring again. If the action did not fix its target, created a new problem, shifted the product's colour, or GrabCut disagreed with the mask, it is reverted and that tool is ruled out.

It ends with one of four answers, each with evidence: ready to publish, fixed automatically (with the corrected photo), reshoot (with a concrete tip such as "move closer"), or a person should decide (with Approve / Reject buttons). Scene and lifestyle photos are recognised and never cut out.

How we built it

  • OpenCV 5.0 for all perception and all tools; NumPy; Python 3.12.
  • AWS Lambda on arm64 (Graviton) with a public Function URL, OpenCV 5 and NumPy as Lambda layers, CloudWatch Logs (one JSON line per request, no pixels), an hourly invocation cap that switches the function off through CloudWatch, SNS and a kill-switch Lambda, and an AWS Budgets alert. Everything serves inside the AWS free tier.
  • COOL: the same workload ran on EC2 c8g.large (Graviton4) from the COOL 3.1 Marketplace AMI, side by side with the stock opencv-python-headless wheel on the same instance, two runs of three alternating rounds each, with a per-stage profile.
  • Evaluation set built from Amazon Berkeley Objects (CC BY 4.0): 252 clean catalog photos x 13 controlled defects = 3,276 labelled photos, split into dev (tuning) and held-out test; plus 106 real catalog photos labelled by hand.
  • An optional Amazon Bedrock (Nova) planner is implemented and unit-tested: it must choose from the legal action list and falls back to rules otherwise. The public demo uses the rule planner to stay in the free tier.
  • Built by one person with an AI coding assistant (Claude Code) for implementation; the agent loop itself runs without it.

Results

  • 89.6 % correct decisions on 1,768 held-out test photos (87.5 % with the strict answer sets written down before any result). The same rules and tools without the re-measure step: 85.2 %, and 255 of its 1,032 "fixed" photos still fail the checks.
  • On the 371 test photos where the agent rolled back one of its own actions, it reached the right decision 94.6 % of the time (open loop on the same photos: 73.9 %).
  • 106 hand-labelled real catalog photos: 27 of 33 problem photos stopped (open loop: 11 of 33); 18 of 23 "accept" verdicts correct. The cost: about four in ten good photos still go to a person.
  • 100 % on strong blur, low resolution, dark photos and cluttered backgrounds; weakest on overexposed white products (33 %).
  • COOL on Graviton4: box filter 2.87x, Otsu 1.94x, CLAHE 1.40x faster; whole agent 1.03x. A per-stage profile shows why: perception is about 40 % of the time and about 80 % of it is NumPy, which COOL cannot reach. adaptiveThreshold 0.77x and Canny 0.90x were slower. Decisions identical in all twelve runs.

Challenges

  • A first version "fixed" room scenes by cutting furniture out of context. A scene check on the GrabCut foreground now refuses those.
  • Professional catalog photos looked "noisy" because fabric and print texture fooled the noise estimator; measuring noise on flat regions only fixed it.
  • White products on white backgrounds look like blown highlights. That is still our weakest case (33 % on overexposed test photos) and it goes to a person instead of forcing a reshoot.

What we learned

Measuring after every action is what makes an agent trustworthy: most of our quality gains came from better verification, not from better fixes.

What's next

Per-marketplace policies (Amazon, Etsy, Rozetka), moving the perception math from NumPy into OpenCV primitives so COOL's fastest kernels reach it, and evaluation on real phone photos labelled by sellers.

Try it

Photos in the samples and evaluation: Amazon Berkeley Objects, CC BY 4.0.

Built With

Share this project:

Updates

Submission history