Minarai

An apprentice QA agent that watches your app — OpenCV 5 finds real UI defects, Claude decides what happens next.

Comment

Inspiration

Every release of a mobile app means someone has to eyeball every screen, in every theme, at every font scale, and catch what's broken. Existing visual-testing tools don't close this gap: they're baseline-diff tools that miss defects present since day one, they don't work on Flutter (which renders to a Canvas with no DOM to diff), and they're priced for a QA budget, not a solo developer. Minarai targets that gap directly.

What it does

Minarai watches your app's screenshots and finds real UI defects with OpenCV 5 — no baseline required. It detects low-contrast text (exact WCAG 2.x math, with a computed color fix), Flutter overflow stripes, blur, missing images, text truncation and overlap (via OpenCV 5's rewritten DNN engine running EAST), and build-to-build visual diffs. An agent loop then decides what happens next — file an issue, ask for another capture, or flag for human approval — and re-verifies fixes automatically.

How we built it

The detection pipeline is pure OpenCV 5 (opencv-contrib-python-headless) plus NumPy. It runs identically locally (python -m minarai check) and inside an AWS Lambda container. The full pipeline is deployed and verified end to end: upload a screenshot to S3, and it auto-triggers EventBridge → Step Functions → the OpenCV 5 Lambda → a Bedrock (Claude Haiku 4.5, via a cross-region inference profile) decision Lambda → DynamoDB, with zero manual steps. A Next.js dashboard (deployed on CloudFront + S3) visualizes results with before/after sliders, diff strobing, and the agent's decision trace.

Challenges we ran into

The real deploy surfaced things a spec can't predict: opencv-contrib-python's full build crashes cold-start in a Lambda container over a missing libxcb.so.1 (fixed by switching to the headless package); Bedrock in ap-northeast-1 rejects a bare model ID and requires a cross-region inference profile instead; and pHash — assumed to be a cheap "no visual change" gate — turned out unable to detect a 3%-of-screen color change, so it's used only for screen-identity triage, not gating.

Accomplishments that we're proud of

Benchmarked against naive pixel-diff on 50 seeded test cases, Minarai improves F1 from 0.309 to 0.891 with 24x fewer false positives — while matching the naive tool's recall. In the field, it found a real WCAG contrast defect on a live app's paywall screen (the core subscription copy, nearly unreadable) — the exact kind of day-one defect a baseline-diff tool structurally cannot catch.

What we learned

That the biggest false-positive sources in real usage (OS status bar chrome, intentional flat-color design cards) never show up in a synthetic benchmark — only real field testing surfaces them, and fixing them measurably cut noise without losing the true positive that mattered.

What's next

Wiring COOL (Cloud-Optimized OpenCV Library) on Graviton for a same-instance benchmark against plain OpenCV 5, and running Minarai against the author's other three production apps to build a larger field-evidence corpus.

Built With

Share this project:

Updates

Submission history