-
-
A real frame from my front door via the Ring Partner API, watermarked by Ring. Classified locally in 4.7 s: no person.
-
One brain in the building. Ring sees, the box decides, Alexa+ and Fire TV tell you. The model reports; rules decide.
-
20 real frames: 90% accuracy, 3/3 recall, zero misses. Both errors were mail carriers flagged for review.
-
Running now: linked Ring account, two cameras, seven services, polled every 30 s. Alexa+ and the lobby screen read one store.
-
Privacy is architectural: no image reaches Alexa+, the lobby shows counts not faces, every tool is read-only.
-
MIT licensed. Eleven-entry friction log with fixes, and a PR adding HMAC verification to Amazon's own sample.
-
Front Desk: a doorstep agent for a law office. Ring sees it, a local model decides, Alexa+ tells you. Nothing leaves.
-
Front Desk: Ring sees, the box decides, Alexa+ tells you.hing leaves.
Inspiration
I'm a litigator admitted in the Southern and Eastern Districts of New York. A process server comes to a small firm's door and nobody is at the desk. The camera records it, the clip sits in a folder, and nobody opens that folder until a deadline has already started running.
"Motion detected" is not an answer. The firm needs to know what was at the door — and a law office cannot send its doorstep to a cloud AI service to find out. Visitors to a lawyer are identifiable, and who visits a lawyer is itself sensitive.
So the model has to live in the building. Ring is the eye, a box in the office is the brain, and Alexa+ and Fire TV are how the answer reaches you.
What it does
A Ring event arrives. The receiver acks in 27 ms and hands off to a worker. The worker fetches the frame, downscales it, and runs a 30B-parameter vision model (Qwen3-VL) on an AMD Strix Halo box via llama.cpp/Vulkan.
The model is never asked "is this a process server?" It cannot know that. It reports observables — a person, facing the door, holding flat papers, no courier markings. A deterministic rules layer turns those into a disposition: delivery, visitor, passerby, parcel_removed, or possible_service. Wrong answers are debuggable, the rules are auditable, and swapping models can't silently change legal behaviour. possible_service means a human should look — never that service was effected.
Three surfaces read one store:
- Alexa+ — a self-hosted MCP server (Streamable HTTP) with five read-only tools: who came by?, what needs review?, event detail. No image ever crosses that boundary.
- Fire TV — a lobby screen showing status and counts. Never faces. LAN-only.
- AWS — fleet health telemetry to CloudWatch. Uptime, latency, disk, service state, an opaque BoxId. Never a frame, transcript, or name.
How we built it
Python, stdlib-heavy by design. frontdesk.py verifies Ring's HMAC over the raw body (X-Signature, UTF-8 key — both confirmed against Ring's knowledge MCP), dedupes on request_id, and acks fast. poller.py is the alternative Ring's docs sanction: poll the Event History API every 30 s and fetch frames via the media endpoint, so no inbound endpoint is required at all. Account linking is Ring's one-way nonce flow — mandatory operator sign-in, HMAC nonce matching, the App-Integrations POST and PATCH.
The vision model runs on hardware in the house: 32 cores, 128 GB unified memory, Vulkan on the integrated Radeon 8060S (no ROCm — RADV was already present). Measured against the same binary on CPU: 2.0× prompt processing, 2.3× generation.
Seven services under systemd. Everything survives a reboot. Cloudflare tunnel for the public webhook endpoint; the dashboard is deliberately not tunnelled.
Challenges we ran into
Account linking is a two-step flow the console doesn't explain. Connecting an account in the Developer Console only registers a sandbox test account. The real linking starts in the Ring App Store. I built the OAuth-server model first because the console's form matches it — then found the docs say to skip that model for new work. Built it twice. Ring support confirmed the sequencing.
Cloudflare fronts oauth.ring.com and returns error 1010 to Python's default User-Agent. Every token exchange failed with a 403 until a real UA was sent. Nothing in the docs mentions it.
The signing key is issued base64-encoded, and the docs don't say whether to decode it. Neither do they name the signature header. Ring's knowledge MCP answered both: X-Signature, UTF-8 bytes, hex for signatures and URL-safe base64 for nonces. Same key, two encodings, and the docs warn not to mix them.
History events carry no thumbnail_url. Only webhooks do. The poller was recording events blind until frames were fetched via the media endpoint with a timestamp — and a single timestamp can land on a gap, so it retries at several offsets.
The GPU wasn't being used at all. The service account wasn't in the render group, so /dev/kfd couldn't be opened. No error. Everything silently ran on CPU for weeks.
Accomplishments that we're proud of
It runs on real cameras. Linked Ring account, front door and backyard, frames pulled through the Partner API with Ring's own watermark reading Partner: Front Desk. Nine hundred events processed unattended.
Measured: 18/20 (90%), possible_service recall 3/3, zero misses. Both errors were mail carriers flagged for review — false alarms, never misses. For a law office that is the right way to be wrong.
The ack-first, idempotent receiver matched Ring's requirements before reading them. Ring requires 200 within 5 s and idempotency on request_id. We ack in 27 ms.
Privacy is architectural. A test asserts no image fields leak to the voice surface. Every MCP tool is read-only. The lobby screen cannot show a face because it was never given one.
What we learned
That the evidence often isn't in the frame. A plain-clothes mail carrier holding letters looks exactly like a process server holding papers. We tried expanding the uniform definition and adding courier observables — net effect zero. The answer is more context (the marked truck ten seconds earlier, arrival history), not a better prompt. Tuning harder would teach the model to guess, and guessing is how misses happen.
That silent failures are the expensive ones: a GPU group membership, a User-Agent string, a sandbox account that looks linked. None raised an error.
And that a vendor's knowledge MCP server is worth querying before assuming anything. It settled four open questions in an afternoon.
What's next for Front Desk — A Doorstep Agent for a Law Office
More possible_service frames and night/IR frames to bound recall. Motion-zone filtering so the classifier wakes only for the doorstep polygon, not the street. Live one-way audio via WHEP — the box can hear the door, which a still frame can't. And a pilot cohort of New York firms beyond my own.
MIT licensed, with an eleven-entry friction log and a PR to AmazonAppDev/ring-api-helloworld adding the HMAC verification the sample is missing: https://github.com/banksythequantLab/front-desk
Log in or sign up for Devpost to join the conversation.