Prism

Ring doorbell alerts for people who can't hear the chime. Bedrock describes the snapshot, and an explainable Signal Score turns it into a card, vibration, tones or a push alert.

Comment

Try it

Live demo: https://d2w68k35faj3ab.cloudfront.net (no sign-up, phone or laptop)

  1. Press Send a test visitor. The alert arrives in a few seconds with its Signal Class, score and reasons. Notable and Urgent alerts also play low tones (turn your volume up). Keep pressing: the first five tell a short story ending in an Urgent night-time visitor, then 13 more cover people, packages, vehicles and animals, by day and at night.
  2. Under Catch up, press What did I miss?

Runs on AWS (EC2 behind CloudFront, Bedrock through an instance role) with simulated Ring events. Everyone trying it shares one household.

Inspiration

Ring doorbells alert through sound and app chimes, which don't reach people who are deaf, hard of hearing or DeafBlind. The usual workaround is separate strobe or bed-shaker hardware that flashes the same way for a package, a cat or a stranger at midnight, with no idea what the camera saw.

What it does

AI describes what the camera saw. Plain code decides how urgent it is, and shows why.

Prism turns each Ring event into an alert you can see and feel:

  • Understands the snapshot: Amazon Bedrock (Claude Haiku 4.5, multimodal) classifies it as a person, package, vehicle or animal and writes a one-sentence description, which doubles as alt text.
  • Scores it with an explainable Signal Score (0–100) and sorts it into Routine, Notable or Urgent. Every factor is shown: category, model confidence, late night, repeat visit, known visitor, quiet hours and household feedback.
  • Delivers it the way it matters:
    • Routine gets a visual card.
    • Notable adds a vibration pattern.
    • Urgent adds a longer, unmistakable pattern and a Web Push notification that lands even when the app is closed.
    • The patterns are designed to be told apart by feel alone.
    • Notable and Urgent alerts also play low tones in the same rhythm, generated in the browser, so a laptop or an iPhone that can't vibrate still carries the pattern. Sound can be switched off on each device.
  • Remembers repeat visitors within a session (Titan Text Embeddings V2 + pgvector), so someone ringing twice doesn't escalate. Identity-free by default.
  • Personalizes without an account system: household time zone, quiet hours (which turn down packages, vehicles and animals but never a person at the door), per-class vibration overrides, and thumbs up/down feedback that gradually adjusts scoring weights. Naming a known visitor is strictly opt-in and stored only in the household's own database.
  • Catch up ("What did I miss?"). A Strands agent on Amazon Bedrock looks through your recent alerts itself and answers in a few plain sentences, short enough for a braille display. You can also ask your own question, such as "Did a package come today?".
  • Respects Ring privacy zones. Prism reads each camera's privacy zones from the Ring API and blacks them out before a snapshot is classified, embedded or shown.
  • Never misses an event. If the backend was down or a webhook never arrived, Prism reads each device's Ring event history and catches up. Recent events still alert; older ones go quietly into history and the catch-up.
  • Your data, your call. Alert history is deleted after 30 days. Settings > Your data deletes it now, after a confirmation step. docs/PRIVACY.md lists what's stored and what leaves the server.
  • Try it in one tap. The Send a test visitor button sends simulated Ring events through the real pipeline: 18 visitors across people, packages, vehicles and animals, by day and at night.
  • Works with Ring's Partner API as documented: it receives Ring's webhooks, downloads each event's snapshot from the Ring API, names the device behind each alert ("Front Door · Person · 95% confidence"), and links a household's Ring account with Ring's one-way account linking.
  • Honest about what's been tested: I don't have Ring hardware, and the Developer Playground's device has no events or recordings and no way to trigger one. So live Ring events haven't been tested. The demo uses a simulator that sends Ring's exact webhook format, signed the same way, while the device name on each card comes live from the Ring API for the Playground device.

How I built it

  • Monorepo (TypeScript, npm workspaces):
    • prism-alert-engine: a Ring-agnostic, MIT-licensed scoring and channel package.
    • prism-backend: Express, WebSocket, Postgres + pgvector, Bedrock and the Ring integration.
    • prism-companion-web: a React PWA.
  • AWS services:
    • Amazon Bedrock Runtime InvokeModel with Claude Haiku 4.5 through the US cross-Region inference profile (from us-east-2), at temperature 0 so the same snapshot is always described the same way. The catch-up agent uses the same model through the Converse API.
    • Amazon Titan Text Embeddings V2 (1024 dimensions) for repeat-visitor memory.
    • A least-privilege IAM policy that grants only bedrock:InvokeModel on exactly those models (infra/aws/).
    • The public demo is deployed with one CloudFormation template (infra/aws/demo/prism-demo.yaml): EC2 (Ubuntu 24.04) running Postgres, backend and app; CloudFront for HTTPS and WebSockets; an instance role with least-privilege bedrock:InvokeModel, so no AWS keys are stored anywhere; Session Manager instead of SSH; a security group open only to CloudFront's origin-facing prefix list.
    • The stack only reports success after the server has classified its first events with Bedrock (a CloudFormation wait condition signalled from the instance).
    • Deployed from AWS CloudShell, so no admin access keys were ever created.
  • Ring, following Ring's Partner API docs:
    • Webhooks: Ring's v1.1 JSON:API format, verified with the HMAC-SHA256 X-Signature header. button_press and motion_detected (with Ring's Smart Alerts sub_type: human, package delivery, vehicle) become alerts. Redelivered events are skipped, and replies follow Ring's retry rules (200, 400 for malformed, 500 to have Ring retry).
    • Snapshots: Ring's webhooks carry no image, so Prism downloads the frame with POST /v1/devices/{id}/media/image/download. Right after an event the recording may not be ready, so 416 and 425 are retried on Ring's backoff schedule, with jitter, but capped at about 20 seconds rather than Ring's suggested 2 minutes, so the alert isn't held up. If there's still no picture, the alert goes out at once from Ring's own event type ("Someone pressed the doorbell"), and the app can load the picture later.
    • Device names: GET /v1/devices, cached and best-effort, shown on every card.
    • One-way account linking: Prism hosts the Token Exchange URL and the Account Link page Ring redirects to. It exchanges the code, matches Ring's HMAC nonce to the right account, confirms the link through app-integrations, refreshes tokens, and deletes them when Ring reports the app was removed.
    • A signed event simulator that sends the same v1.1 webhooks, so anyone can run the full pipeline without Ring hardware.
  • An agent where the model has decisions to make, and a plain function chain where it doesn't:
    • The alert pipeline (classify, score, decide channels, deliver) runs the same four steps for every event, so it's a linear chain with a single Claude call to classify the snapshot. That keeps alert latency predictable.
    • The catch-up summary is a Strands Agents agent, using the TypeScript SDK with Claude Haiku 4.5 on Bedrock. It has four read-only tools: get_current_time, list_new_alerts, list_alerts and count_alerts. The model decides which tools and filters answer the question, such as a time window, a Signal Class or a category, and whether counts or individual alerts answer it. What it would otherwise have to work out from timestamps is decided in code: which alerts are new since the household last looked, the opening count of new alerts, and whether each alert happened during sleep hours (the household's quiet hours, or 10 PM to 7 AM).
    • AgentCore isn't used. The agent is a short request-and-response call inside the existing Express backend, so a separately hosted runtime would add deployment without adding capability. It's the next step if the agent moves to a hosted deployment.
  • Accessibility was tested from the start:
    • axe-core runs on every component and page state, with zero violations;
    • WCAG contrast is computed from the real palette (text 6.7:1–17.9:1);
    • a keyboard-only walk confirms focus order and no traps;
    • the live alert card is an assertive live region kept short for screen readers.

For Ring track reviewers

  • README tables: every Ring API call and every AWS service, linked to the code.
  • npm run forged-webhooks: sends an unsigned and a wrongly signed Ring webhook to the live demo; both are rejected with a 401.
  • docs/RING_READINESS.md: Prism checked against Ring's certification requirements, with honest "to do" items (not yet run against a live Ring account; no hosted privacy policy or terms; design checklist not yet gone through).

Measured results

  • Classification: 33/34 (97%) on 34 real labeled photos chosen to be hard: low light, motion blur, partial frames, look-alike categories. Person 9/10, package 8/8, vehicle 8/8, animal 8/8. The only miss is a person walking a dog that dominates the frame.
  • Pipeline latency (classification + scoring + channel decision): mean 1804 ms, p50 1635 ms, p95 3082 ms.
  • Delivery: WebSocket delivery averages 0.068 ms (localhost, 1,000 events × 5 clients).
  • Repeat-visitor matching: visitor signatures separate cleanly. Same-visitor similarity is at least 0.634, different visitors at most 0.569, and the threshold is set at 0.60 from those measurements.
  • Catch up: 30/30 answers with the facts and format right (10 questions, each asked 3 times, on a fixed household day with known answers), median 2.7 s. The first version got 26/30; the fixes were tuned on these same questions, so this shows the fixes work rather than accuracy on new questions. Method and all four runs: docs/ARCHITECTURE.md.
  • Tests: 486 automated tests across the three packages (engine 19, backend 351, companion app 116), as of Oct 18.

Challenges

  • Repeat visits were missed. Bedrock's default temperature described the same photo differently each time, and the first threshold (0.85) sat right at the average same-visitor similarity. I built a measurement tool (npm run eval:repeat), moved matching to a short "visitor signature" (clothing, colors, carried items), and set temperature to 0. Repeat visits are now recognized reliably.
  • Large photos failed with a generic error. A few large camera exports failed with a generic InvokeModel error that didn't say why. Resizing them fixed it; a descriptive error would have saved time.
  • Ring's account-linking model was the reverse of what I assumed. I first built a partner-initiated OAuth redirect. For apps installed from the Ring AppStore, Ring's docs describe one-way linking instead: the partner hosts the Token Exchange URL and Account Link page, and Ring calls them. I rebuilt it to match, including the HMAC nonce check.
  • The catch-up agent ran long and wandered off the question. I built an eval with known answers (npm run eval:catch-up). The first run found answers over the 80-word limit and an overnight answer that mentioned the evening before. Fixing those in code (Routine alerts kept apart, a sleep-hours filter, shorter time labels, and a one-time rewrite of any answer that's too long) took it from 26/30 to 30/30. Run 3 passed while an answer drifted onto unrelated alerts, because nothing checked for that, so I added the check and fixed the prompt.
  • Quiet hours turned down the visitor that mattered most. Testing the live demo, a stranger at 11 PM scored 69, one point short of Urgent: quiet hours took 10 points off everything, cancelling most of the late-night factor. Quiet hours are when someone deaf is asleep, so they now turn down packages, vehicles and animals but never a person.
  • Testing Ring without Ring hardware. The Playground device lists fine, but it has no events or recordings and nothing to trigger one, so snapshot downloads return "media not found". I built the simulator to send Ring's exact signed webhooks, and kept a fallback so an alert never waits on a missing picture.

Accomplishments

  • 33 of 34 labeled photos (97%) classified correctly, including low light, motion blur and look-alike categories. Median 1.6 s per event.
  • 30 of 30 catch-up answers right on a fixed test day, up from 26 of 30, with each fix measured.
  • Repeat-visitor threshold (0.60) above every different-visitor similarity measured (max 0.569).
  • WebSocket delivery 0.1 ms at p95 locally (5,000 deliveries).
  • Zero axe-core violations across every component and page state; manual VoiceOver pass, all 10 checks.
  • Distinct alert channels (card, vibration, sound and push), each with an explainable reason.
  • A standalone open-source scoring engine.
  • An upstream documentation contribution to web-push.
  • An honest evaluation harness for every number above.

What I learned

  • Measure before tuning: the repeat-visitor threshold looked reasonable and was wrong.
  • An eval only catches what it checks: one run scored perfectly while an answer drifted off-topic, until I added a check for it.
  • Accessibility work is cheapest at the start.
  • Explainability matters more than a single number when the alert decides whether someone opens the door.

What's next

  • Run against a real Ring doorbell: live webhooks to a deployed backend, account linking from the Ring AppStore, and real snapshots.
  • Smartwatch haptics.
  • A manual VoiceOver/NVDA pass with Deaf and DeafBlind users.

Built With

Share this project:

Updates

Submission history