ApertureGrade

Home page
OpenCV analysis
Digital Passport + Fraud Detection
Live Camera

Inspiration

Circular commerce runs on photos. Before a laptop can be refurbished, resold, or routed for parts recovery, someone has to capture three views — front, rear/ports, and label — clearly enough to inspect. In the real world, that means blur from shaky hands, glare on serial stickers, shots taken in a dark corner, and incomplete sets where one required angle never got taken.

American Circular gave us a dataset built from their own scrap equipment and a clear rubric: detect the problems, explain what's wrong, and tell the operator exactly what to reshoot. We built ApertureGrade because the gap isn't "take a photo" — it's take a photo that's actually usable downstream. Every bad capture wastes triage time, blocks resale, and slows the circular loop.

What it does

ApertureGrade is a capture coach and quality gate for electronic device photography.

Live Camera mode walks an operator through front → rear → label with voice coaching, MediaPipe device tracking (corner-bracket AR overlay), a stability ring that auto-captures when the frame locks, and immediate retake loops when a shot fails. When blur, underexposure, glare, framing, or label obstruction is detected, ApertureGrade shows a side-by-side rejected vs. accepted compare panel and speaks specific fix-it guidance — not a generic error.

Upload mode evaluates full photo groups against the American Circular practice sets (SET-001 through SET-009) and the 34-image catalog. For each photo you get usable, retake, or needs_review, with issue codes and retake instructions. Missing views are reported separately from bad photos — a full set can still contain shots that need a retake.

When all three required views pass, ApertureGrade issues a Digital Product Passport: condition grade, device class guess, refurb route value, CO₂ savings vs. landfill, component callouts, fraud/view-mismatch flags, and a disassembly playbook — with a QR code for the passport ID.

Two modes for demos: Demo voice (deterministic scripted retakes for a reliable judge video) and Operator (real OpenCV QC on every live capture).

How we built it

Frontend: Next.js 16 (App Router), React 19, TypeScript, Tailwind CSS, and shadcn/ui, organized in a feature-driven /features/capture module with Zustand for session state.

Live capture stack:

MediaPipe EfficientDet Lite0 (@mediapipe/tasks-vision) for on-device object detection — filters to person/hands and tracks the device for AR corner brackets
Custom hooks for stability gating, scripted demo sequencing, and the full capture session lifecycle (coach → lock → capture → QC → retake → passport)
Pre-rendered ElevenLabs MP3s in public/audio/aperturegrade-demo/ — no TTS API at runtime, Cognifit-style sequential playback

Backend: Python FastAPI + OpenCV in server/aperturegrade/, proxied through Next.js at /aperturegrade-api/* so phone demos work over HTTPS/ngrok without hitting Safari-blocked ports.

QC engine (qc.py): Deterministic checks aligned to the challenge photo guide — Laplacian blur, exposure histograms, glare hotspots, framing coverage, DEMO-label obstruction — with softer live thresholds for webcam captures vs. uploaded files.

Insights layer (insights.py): Passport synthesis, device-class routing, condition grading, circular impact modeling, and harvest playbooks.

Run: bun dev starts Next.js and uvicorn together via concurrently. One command, local analysis, no paid GPU, no custom model training, no ACS production data.

Challenges we ran into

Safari and mobile camera access. iPhone Safari blocks many dev ports (including :8000). Direct API calls from the phone to localhost failed silently. We routed all analysis through the Next.js proxy on the same origin and documented ngrok + HTTPS workflows for live demos.

Retake loops that never closed. Early live QC was too strict for webcam noise; operators would pass stability, capture, fail QC, and get stuck. We softened live thresholds, reset the stability gate on each retake, and scoped blur detection to the detected device ROI instead of the full frame.

Demo reliability vs. real QC. Judges need a deterministic story every time; operators need honest OpenCV feedback. We split Demo voice (scripted blur → dark → glare → framing sequence with timer-based lock ring) from Operator mode (real analysis on every capture) instead of trying to make one path do both.

Autoplay and audio timing. Browser autoplay policies blocked welcome clips that started after async camera permission. We prime audio on the user's first tap and chain MP3 cues only after each clip finishes.

Separating "missing" from "bad." The rubric treats a missing rear shot differently from a blurry rear shot. We kept those as distinct findings in both the API response and the UI so operators don't confuse "go back and take this view" with "retake what you already have."

Accomplishments that we're proud of

A working end-to-end prototype — live camera, upload workflow, practice-set scoreboard, and passport — not just a upload-only MVP
OpenCV QC validated against all 34 practice photos and 9 full sets in the American Circular dataset
Actionable retake coaching with view-specific voice lines (blur, dark, glare, framing) and a visual before/after compare panel
Digital Product Passport that closes the loop — capture quality isn't an abstract score; it unlocks refurb routing, CO₂ impact, and a harvest playbook
Phone-ready live demo with AR device tracking, auto-capture ring, and proxy architecture that actually works on iPhone over ngrok
Zero runtime API cost — local OpenCV, on-device MediaPipe, pre-rendered voice clips
Deterministic demo mode so remote judges see the same story every time: reject blur → accept front → reject dark → accept rear → reject glare/framing → accept label → passport

What we learned

Computer vision for operators is a UX problem, not just a metrics problem. Laplacian variance numbers mean nothing to someone holding a laptop in a warehouse. What matters is: "That photo is blurry — hold steady and try again," spoken at the right moment, with a visual compare of what failed.

Live camera QC needs different thresholds than uploaded photos. Webcam frames are noisier, smaller, and more motion-prone. One threshold set for both modes creates false retakes or missed defects.

The circular economy needs trust at the capture layer. Downstream resale, routing, and compliance all assume the photos are honest and complete. Building explicit separation between missing views, retake-worthy shots, and passport-ready sets mirrors how real triage workflows think.

Hackathon demos need a script. Real AI is probabilistic; judge videos can't be. A scripted demo path alongside an operator mode let us show the product story reliably without faking the underlying QC on uploads and practice sets.

What's next for ApertureGrade

Automatic view classification so operators don't have to tag front/rear/label manually on upload
Tighter integration with ACS triage workflows — batch upload from floor stations, export passport JSON to inventory systems
Stronger device identity from label OCR (serial/model extraction) with clear fraud flags when photos don't match the claimed device
Operator analytics dashboard — retake rates by issue type, average captures-to-passport, floor-level quality trends
Offline-first mobile app (React Native / Expo) for warehouse capture without depending on a laptop proxy
Expanded condition grading using component visibility from rear and label shots, not just capture quality
Multi-language voice coaching for international refurb operations