Northbridge Watch

Quiet-by-default home vision: classical OpenCV 5 spots packages, spills and empty fridges, and a small policy agent decides whether to stay silent, log, notify, or ask a human.

Comment

Inspiration

Home cameras alert on everything. After a week of "motion at front door" for an empty porch, people mute them, and then they miss the one alert that mattered. Our test household, Avery & Morgan (pseudonyms), wanted the opposite: a camera that stays quiet unless something really changed, and that asks a person when the stakes are real.

What it does

Northbridge Watch looks at porch and kitchen frames and turns what it sees into one of four actions:

  • noop: calm scene, nobody is pinged
  • auto_log: a low-stakes event, logged quietly
  • notify: one person, one short message (e.g. "A package arrived on the porch")
  • human_required: a safety event such as a floor spill; a human must acknowledge

It recognises five scene types: calm porch, package at the door, several packages blocking the door, a kitchen floor spill, and an empty vs. full fridge shelf. Quiet hours (22:00–07:00 UTC by default) hold back non-urgent notifications. Every run shows the full trace: perception, decision, speakable message and the exact action JSON.

Try it: open https://northbridge-watch.vercel.app, pick a scene, press Run live OpenCV 5 pipeline, or upload your own image. Uploads are processed in memory and never stored.

How we built it

  • Perception (OpenCV 5.0.0, classical, no learned model): resize to a fixed 640×480 working size → HSV conversion, inRange masks and morphologyEx to isolate cardboard and liquid → findContours, approxPolyDP and convexHull solidity to tell a box (compact, four-sided) from a spill (irregular, low solidity) → Sobel gradients on a normalised image plus Canny to find shelf lines and floor tiles → an occupancy gate (4% foreground) to separate an empty fridge shelf from a full one.
  • Policy agent: a small deterministic PolicyEngine maps detections and confidence to noop / auto_log / notify / human_required, with quiet hours and escalation rules (spill → always a human; two or more packages → doorway blocked → escalate). An AgentOrchestrator runs perception → policy → actions and records a trace.
  • Live demo: the same Python package runs as a Vercel serverless function (/api/analyze, /api/health) behind a static front end. Measured server-side pipeline time on the live app is about 10 ms per frame (9.8–13.7 ms over 15 warm requests on 2026-10-25); browser round trip was 67–124 ms.
  • AWS: an AWS SAM template (API Gateway HTTP API → Lambda on arm64 → S3 event log + SNS notifications) with a tested Lambda handler and a /health route. It is not deployed yet; make deploy-aws deploys it and wires the URL into the Vercel app.
  • Evaluation: a reproducible synthetic benchmark (make eval): 7 randomised scene families × 40 images each, under 5 conditions (clean, photometric shift, blur + noise, JPEG q30, half resolution), with held-out test seeds never used for tuning, plus a small out-of-domain set.
  • Engineering: 26 unit tests (make test), a one-command demo (make demo), and a technical report with method, architecture, evaluation, failure cases, limitations and responsible use.

Challenges we ran into

Building the benchmark exposed three real bugs in our first detector (v1): every full fridge was labelled "empty", every package was missed at half resolution because the area thresholds assumed full-size frames, and spills disappeared under exposure changes. Fixing them (a fixed working resolution, normalised gradients, an explicit occupancy gate) gave v2. We also found that uploads were being written to disk, which contradicted our privacy goal, so they are now handled in memory only.

Accomplishments that we're proud of

Label accuracy on the held-out synthetic test split (280 images per condition), v1 → v2:

Condition v1 v2
clean 82.9% 96.8%
photometric shift 80.7% 92.1%
blur + noise 83.9% 97.5%
JPEG q30 83.2% 96.8%
half resolution 44.6% 96.8%
out-of-domain, 120 images (coffee spill, white mailer, night package) — 33.3%

Spill recall is 87.5% on clean images and 70.0% under photometric shift. The whole pipeline is explainable: every decision traces back to a mask, a contour and a rule.

What we learned

A small honest benchmark is worth more than a big claim. Our v1 looked fine on hand-picked demo images and failed systematically once we varied resolution and lighting. With classical OpenCV the failure modes are easy to read, which made them easy to fix and easy to explain.

Limitations

  • Synthetic data only. All numbers come from our own scene generator, not real home photos. Real photos will often come back "calm". Treat the numbers as a sanity check, not real-world accuracy.
  • Colour-dependent. Coffee-coloured spills and white mailers are missed (0/40 each), so out-of-domain accuracy is 33.3%.
  • Lighting. Spill recall drops to 70% under photometric shifts.
  • AWS not deployed. The SAM stack is written and tested but not live; the live demo runs on Vercel.
  • Single frame. There is no temporal tracking yet, so a package that is delivered and then picked up is two independent frames.

Responsible use

No faces or people are detected or identified; the detector only looks for packages, liquid and shelf occupancy. Uploaded images are processed in memory and not stored. Safety events always go to a human rather than being auto-resolved. The household names in the demo are pseudonyms.

What's next

  • Deploy the AWS stack (make deploy-aws) so events land in S3 and notifications go out over SNS.
  • Collect a small, consented set of real porch and kitchen photos and report real-world accuracy next to the synthetic numbers.
  • Add colour-independent spill cues (specular highlights, texture change against a reference frame) to fix the coffee-spill failure.
  • Add temporal smoothing across frames to cut false alerts further.

Links

Built With

Share this project:

Updates

Submission history