AdLint Agent

An OpenCV 5 agent that checks ad creatives before launch: it reframes each one for every placement, re-checks its own output, and asks a human when it can't fix it. Live on AWS Lambda.

Comment

Inspiration

I work in programmatic advertising. One campaign runs the same creative in many placements: 9:16 Stories and Reels, 1:1 and 4:5 feed, 16:9 YouTube, 300x250 banners. Each one has its own shape and its own zones covered by the platform's UI (profile bar, caption, buttons, skip button). Today people resize by hand or with a centre crop, and the same mistakes reach live campaigns again and again: a face cut in half, the logo under the "Shop now" button, a headline under the caption, a logo too small to read on a banner, a dark or soft master nobody flagged.

AdLint Agent sits between the design hand-off and the ad-server upload. It prepares every placement it safely can, and sends the rest to a human with a reason and an annotated preview.

What it does

Give it a creative (image or short video), an optional brand logo and the placements you will buy. For each placement it looks, decides, acts, and looks again at its own output:

  1. Perceive the master (OpenCV 5): text (DB text detector), faces (YuNet), main subject (spectral-residual saliency), logo (ORB + RANSAC homography), sharpness and exposure.
  2. Fix the master if it can: dark or flat -> exposure fix; mildly soft -> sharpen. Then perceive again.
  3. Per placement: resize or smart_crop, then render the candidate and run perception on the output: is a face, the logo or the headline cut? Is any of it under this platform's UI? Is the logo still recognisable at this size? Is the headline tall enough to read?
  4. What it sees decides the next tool call: smart_crop(priority) -> fit_into_safe_zone -> back to a tighter crop if shrinking made text unreadable.
  5. When nothing fixes it, it escalates: the least-bad version is saved as _PROPOSED with the reasons, for a human to approve.

The loop is bounded (4 steps for the master, 6 per placement) and every step is written to trace.json: observation -> decision (with reason and which planner made it) -> tool call -> new observation.

A real trace from the demo (16:9 master, logo hugging the edge):

step placement saw decided after re-perceiving the output
6 reels_9x16 aspect mismatch smart_crop(all) face, logo and text cut, text under UI
7 reels_9x16 same smart_crop(priority) still cut
8 reels_9x16 same fit_into_safe_zone no issues -> accept
17 display_300x250 aspect mismatch smart_crop(all) logo cut, logo under UI, text cut
18 display_300x250 same smart_crop(priority) face cut, logo cut
19 display_300x250 same fit_into_safe_zone logo not re-detected (55 px wide)
20 display_300x250 logo not re-detected escalate PROPOSED asset + annotated preview for a human

Two planners, one interface: a deterministic rule policy (default, no network) and an optional LLM planner (OpenRouter, DeepSeek V4.1 Flash, JSON mode). The LLM only chooses among the vision tools, never sees pixels, and any invalid or repeated answer falls back to the rules. The trace records which planner made each decision.

Video: scene cuts (HSV-histogram Bhattacharyya distance), dark and blurry frame share, a first-3-seconds check, and one reframe per shot so the crop can change at a cut.

How we built it

  • OpenCV 5 (opencv-contrib-python-headless==5.0.0.93) for all perception and all actions: cv2.dnn.TextDetectionModel_DB, cv2.FaceDetectorYN, cv2.saliency, ORB + BFMatcher + findHomography, Laplacian sharpness, CLAHE, VideoCapture / VideoWriter.
  • Python, FastAPI single-page web app, CLI.
  • AWS: Lambda (python3.12, arm64 / Graviton, 3 GB), public HTTPS Function URL, private S3 bucket for every run's assets (presigned URLs, 7-day lifecycle), least-privilege IAM role, CloudWatch logs. Reserved concurrency and upload caps as a cost guard. aws/deploy.sh is idempotent, aws/teardown.sh removes everything, every resource is listed in AWS_RESOURCES.md.
  • Measured on AWS: a 4-placement image check takes 3.8 s warm / 6.8 s cold; an 8-second 1080p video takes 60 s. Idle cost is zero.

Evaluation

Fresh test set of 60 labelled creatives x 5 placements (made after the last code change; synthetic, built from CC0 photos with known defects and ground-truth boxes):

  • Reframing: a plain centre crop keeps everything important in view and clear of platform UI 10% of the time; the agent does it 97% of the time.
  • Detection: blur P 0.90 / R 0.95 · dark 0.94 / 1.00 · text-heavy 1.00 / 1.00 · logo 0.96 / 1.00 · faces 1.00 / 1.00.
  • Safety: 6 false accepts out of 300 outputs; 181 of 300 sent to a human.
  • LLM planner vs rules (20 creatives): same final decision on all 100 outputs, 23% fewer steps, but ~20 s per creative instead of 1.7 s.
  • Video: hard cuts detected perfectly on 6 synthetic clips.

Challenges and failure cases (honest)

  • All 6 false accepts kept the face but cropped off 30%+ of the photo: the agent protects the face when there is one, and does not also protect the rest of the subject.
  • Severe vs mild blur is not reliable (precision 0.50). This is the main reason it sends too many creatives to a human.
  • On 300x250 banners only 65% of headlines stay readable; the agent flags these rather than shipping them.
  • Cross-fades in video produce false scene cuts.
  • ORB missed small logos until I pooled reference features from several logo scales.
  • The test data is synthetic, so these numbers show the loop works, not how accurate it is on real campaigns.

Responsible use

Face detection only, never recognition or identity. Every risky change ends in a human approval with a reason. Nothing is logged except run metadata. Uploaded files are deleted after 7 days. The public demo has the LLM planner switched off so it cannot spend money on third-party APIs.

What's next

Protect the whole subject, not just the face. Real campaign data from agencies. Direct hand-off to ad servers (DV360, Meta, TikTok) so approved assets upload themselves.

Links

Built With

Share this project:

Updates

Submission history