ClaimLens

Photo evidence for claims that holds up: OpenCV 5 checks and measures each photo, an agent asks for the next shot and stops for a human, and every decision ships in a tamper-evident evidence pack.

Comment

Problem and users

People filing insurance, delivery or warranty claims are asked to submit photos, and the claim then stalls because a photo is blurry, glare-covered, missing the receipt, or gives no way to tell how big the damage actually is. Claims handlers spend their time asking claimants to retake photos instead of processing claims, and every round trip costs a day.

ClaimLens moves the quality check to the moment of capture instead of after submission:

  • Claimants are guided, in plain language, through exactly the shots needed, with an immediate, specific reason whenever a photo is rejected ("too blurry: hold still, tap to focus, retake", not a generic "try again").
  • Claims handlers / adjusters receive a complete evidence pack only when the agent judges it complete, or an explicit escalation ("needs_human") naming the measurement or the repeated failure that triggered it.
  • Insurers, delivery platforms, warranty desks get a consistent, auditable intake pipeline instead of an inbox of arbitrary phone photos.

OpenCV 5 implementation depth

Every vision step is OpenCV 5, no other CV library, all server-side (opencv-python-headless==5.0.0.93):

  • Quality gates (claimlens/quality.py): sharpness from the variance of the Laplacian, glare from the fraction of near-white pixels, exposure from mean luminance plus a dark-pixel fraction, framing from the largest dilated-Canny connected component, and a minimum resolution check. Five gates, each returning a number, a threshold, and a plain-language reason.
  • Document detection and flattening (claimlens/document.py): Canny edges, contours, cv2.approxPolyDP to the largest convex quadrilateral, cv2.getPerspectiveTransform + cv2.warpPerspective, plus an adaptive-threshold readability pass.
  • Marker-scaled measurement (claimlens/measure.py): cv2.aruco.ArucoDetector (DICT_4X4_50) finds a printed marker of known size, derives millimetres per pixel, and turns a person-marked damage polygon into real-world cm2, with an edge-density texture-contrast signal alongside it.
  • OCR (claimlens/ocr.py): OpenCV 5's own DNN text APIs, cv2.dnn.TextDetectionModel_DB with the opencv_zoo PPOCRv3 detector and cv2.dnn.TextRecognitionModel with the opencv_zoo CRNN recognizer. Weights are SHA-256-verified by models/fetch.py and fetched again at Docker build time, so the container needs no network at runtime.
  • One OpenCV-5-specific detail: ArucoDetector.detectMarkers returns ids as a 1-D array in OpenCV 5 where 4.x returned (N, 1); measure.detect_marker flattens this explicitly, found by writing the tests.

Agentic loop and human control

A perception-decision-action agent (claimlens/agent.py) turns every photo into exactly one of four actions: retake (naming the failing gate), request the next shot, stop for human review, or complete, and writes every transition to a trace with the measurements that caused it. The visual result of one perception step changes what happens next.

Two conditions force a hard stop for a person and cannot be talked past: a damage area at or above 100 cm2 (measured via the marker) always escalates, and three failed retakes on the same shot also escalate. docs/verification_20260905.md exercises both over real HTTP against the live server, plus a genuine glare-gate rejection. The agent never approves or denies a claim; "complete" means the evidence pack is ready for a person. The built-in demo mode ("Run built-in demo") runs the identical loop on synthetic photos, so the full behaviour can be seen with one click.

Evaluation, including failure cases

eval/run_eval.py runs 196 synthetic images with known ground truth and writes docs/evaluation.md; every number below is quoted from that file:

  • Sharp images accepted 20/20; blurred images rejected (kernel sizes 7/15/31) 60/60.
  • Glare gate correct across 5 disc radii 25/25; exposure gate correct across 4 darkness levels 20/20.
  • Documents found 20/20, median max-corner error 1.0 px.
  • Marker detected across marker sizes and polygon areas 27/27; area error mean 1.28%, max 1.69%.
  • Agent loop reaches the expected terminal action in 6/6 scripted scenarios.
  • OCR word recall 100.0%, latency 220 ms per flattened receipt; quality + measurement latency 5.9 ms per 1000x800 image (Apple M5 Pro, OpenCV 5.0.0).

Real failure cases are demonstrated by hand over live HTTP in docs/verification_20260905.md: a glare-gate rejection of a legible document, both stop_for_human branches, and the 400/403/404/409 error paths. Real-photo evaluation has not been run; every number above is synthetic, stated plainly.

AWS delivery

The live, judge-accessible demo runs on Render at https://claimlens-9rnn.onrender.com (free tier; demo mode needs no camera; per-IP rate limiter; OCR weights baked in at build time). The free instance sleeps when idle, so the first request can take about a minute.

The AWS component (ECR, ECS Fargate on Graviton ARM64, S3 evidence store, CloudWatch) is fully scripted in deploy/aws/ (Terraform plus numbered CLI scripts) and documented in docs/aws_deployment.md, with a checklist that brings the identical container up on Graviton in an estimated 10 to 13 minutes. It has not been executed against a live account: no AWS account exists behind this submission, so no AWS-measured number appears anywhere in this entry. deploy/aws/scripts/05_run_cool_benchmark.sh documents the exact before/after procedure for the COOL award using eval/run_eval.py --out.

Responsible use and limitations

  • No facial recognition, no biometrics, no surveillance; every photo is of an item, a receipt, or a printed marker the claimant chose to photograph.
  • No third-party recognition API; OCR and every CV step run locally with OpenCV's DNN module. On the hosted demo, uploads are processed in memory for that session only.
  • Every photo is SHA-256-hashed and every decision timestamped in the evidence manifest, so a pack is tamper-evident.
  • Thresholds are tuned on synthetic textures and need re-validation on real phone photos; framing can flag a flat subject as too small; measurement needs the marker roughly coplanar with the damage; there is no damage classification model. Full list in docs/technical_report.md.
  • All data in the repo and demo is synthetic and clearly fictional.

Try it

Built With

Share this project:

Updates

Submission history