Doorstep

Doorstep turns Ring cameras into alerts a family describes in plain English: the carer came, Mum's not up by 9:30, someone's still on the floor. Peace of mind for a parent living alone.

Comment

Inspiration

When a parent lives alone, their Ring cameras tell the family that something moved, thirty times a day. The alerts get muted, and the one that mattered is missed. The questions a family actually has aren't about motion:

  • Is Mum up yet?
  • Did the carer come, and how long did they stay?
  • Who rang the bell?
  • And the one they dread: has she fallen, with nobody there?

Most of these are about absence and timing, which a motion alert can't express. We wanted a family to write those worries down in their own words and be told only when one comes true.

What it does

Doorstep runs on a household's Ring devices and gives the family one page with three tabs:

  • Set up. Alerts in plain English ("Tell me if there's no movement in the kitchen by 9:30am", "Tell me if someone is on the floor for more than 2 minutes"), plus expected visits: who should come, what they look like and when.
  • Notifications. Every doorbell press, visit and alert, and a digest of the last 24 hours.
  • View. Every clip on one timeline. You can also ask questions like "Did the carer come today, and how long did they stay?" and get answers that cite the clips.

Every alert also sounds the Ring Chime in the house.

How we built it

  • Ring Partner API. Doorstep's client covers:
    • devices and Event History;
    • clip and snapshot download (following the 303 to the pre-signed URL);
    • Chime playback;
    • webhooks, verified with HMAC-SHA256 and deduplicated on request_id.

The same client also runs against Ring's Developer Playground, where it polls Event History, because the Playground sends no webhooks.

  • A Ring simulator (ringsim). The Playground has two scenes, no doorbell press, no webhooks and no Chime. So we built a server that plays a scripted day of real video through the Partner API's own shapes, with signed webhooks, media redirects and a Chime. Doorstep talks to it with the same client; only the base URL changes.
  • vidra, our open-source video RAG library. It reads every clip in one pass, sampling frames and asking several questions at once:
    • a general description;
    • each alert's own question;
    • each visit's question, "does someone matching this description appear?"

The answers are stored as strict JSON and indexed for hybrid search.

  • Decisions are code, not the model. The model only answers narrow questions ("yes / no / cannot tell"), and plain code applies the rules. There are three kinds of rule:

    • It happened: a clip answers yes.
    • Still true after $N$ minutes: Doorstep checks a fresh snapshot every minute, and alerts only if every check since the first sighting $t_0$ said yes:

    A "no" resets the rule. "Cannot tell" neither fires it nor clears it.

    • Didn't happen by a deadline $d$: fires if no clip in the window $[w, d]$ answers yes. An alert about nobody in the kitchen needs no model at all.

  • Ask. An LLM plans the search, and vidra's hybrid search ranks the clips. Dense and keyword rankings are fused by rank, never by raw score. A clip's score is its best-ranked description plus a discounted second, so many weak matches can't outvote one strong one The answer is written only from that evidence and the visit records.

  • Stack. FastAPI and SQLite, server-sent events to the page, OpenAI models through vidra. There are 51 tests, and a rehearsal script plays the whole demo day and checks every alert, visit, ring, Chime, answer and digest.

Challenges we ran into

  • The Playground can't send what a Ring app depends on. Its scenes arrive as on_demand and ignore the event_types filter. Media is refused for times before the token was granted. It has no doorbell, webhooks or Chime. We built the simulator to show the full product through Ring's API shapes, and kept the Playground path working too.
  • "Didn't happen" is hard to ask a camera. It needs a deadline, the day's clips, and waiting for clips still being read before deciding.
  • One frame is not a fall. Bending down looks like falling. The answer was to keep looking and alert only on still down.
  • Matching a carer by description is noisy. The model's guess at gender changed between runs, so descriptions should use clothes, such as a white VOLUNTEER T-shirt. A carer leaving is often described as "at the door", so a later sighting counts as the departure.
  • Bugs that only showed up in a full day. Two examples, both caught while recording the demo:
    • Replaying the day reused Ring event ids, so Doorstep dropped the repeats as duplicates.
    • Clearing the timeline at midnight wiped the kitchen rule's state, so its 9:30 deadline counted as missed again and the alert fired a second time. A deadline alert now only fires within an hour of its deadline:

What we learned

  • Families don't want more motion alerts. They want to ask about absence and timing, and to hear about something only when it's still wrong.
  • Video understanding is good enough for this today, if the model answers narrow, structured questions and the decisions stay in code you can read and test.
  • The sentence on each alert's card is generated from the rule itself, so what the family reads is exactly what runs. That turned out to be the feature that makes plain-English alerts trustworthy.

Built With

Share this project:

Updates

Submission history