Describe My Door

Describe My Door tells blind and low-vision residents who is at their Ring doorbell, in plain spoken words.

When someone presses the doorbell or leaves a package, a Strands agent takes over:

It downloads the Ring snapshot at the event time.
A vision model describes what is visible: clothing, carried objects, packages.
It checks the caregiver's list of expected visitors, matching only by time window and visible clues, never faces.
It announces one short sentence on the resident's phone, and the Ring Chime plays a matching tone.

"This may be Maria and Tom from next door: two people at the door, one in a grey shirt carrying yellow flowers."

The problem

Approximately 7 million people in the United States have vision impairment, including 1 million with blindness (CDC). A video doorbell is a screen. For them, "who's there?" still means opening the door to a stranger or calling someone to look at the camera.

How it works

Ring Partner API:
- Webhook v1.1 intake with HMAC-SHA256 X-Signature verification over the raw body, idempotency on request_id, and a 200 in under 5 s.
- GET /v1/devices, /capabilities, /status for the devices on the account (live in hybrid mode).
- POST /v1/devices/{id}/media/image/download (at_timestamp).
- Chime configurations → customizable_slots → POST /media/audio/playback with the slot's audio_ref.
Strands Agents SDK (TypeScript): four tools (fetch_snapshot, describe_scene, check_expected_visitors, announce). The deterministic guardrails are BeforeToolCallEvent hooks:
- step order
- privacy: no gender, age, race, religion, health or names
- at most 28 words
- category and label consistency
- one announcement per event

When a hook blocks, the agent sees the reason and rewrites. The demo shows a real block and retry. Every tool call and hook verdict streams to the UI.

Accessible companion app: speech synthesis, ARIA live region, vibration, Atkinson Hyperlegible type, a "Repeat (R)" shortcut, and today's history. High-contrast, screen-reader-first layout.

What is real Ring, and what is simulated

We hold a Ring developer account with a private app, and the demo runs in one of three modes (RING_MODE), with every element labeled in the UI:

Hybrid (with a Developer Playground token): device list, capabilities and status are live Ring Partner API calls; the doorbell event, the snapshot and the chime tone are simulated. This is what the demo video shows.
Simulator (the public demo, which carries no token): everything is simulated, using the documented Ring shapes and the same signature-verification path.
Live: every call goes to Ring; needs an account with hardware.

Measured on the Playground with a live token (scope ava.v1:read): one Doorbell Pro on the account; media/image/download returns 403 TIME_RANGE_NOT_AUTHORIZED for every timestamp; configurations.audio.customizable_slots is null and audio playback returns 400; event history is always empty; and the Playground's simulate buttons only open a WHEP live-view session in the browser rather than delivering an event to the app. Details in FRICTION_LOG.md #6 and #7.

Privacy by design: no face recognition, no identity guesses, snapshots not stored. All people in the demo are fictional; the photos are Pexels-licensed stock.

Live: https://describe-my-door.vercel.app · Video: https://youtu.be/0lSdXD05W7o · Code: https://github.com/chinesepowered/hack-amazon4