Hearthbeat

Care that asks before it alerts: Ring sensors, not cameras, learn an older adult's daily rhythm, and Alexa+ checks in with her first.

Comment

What it does

Hearthbeat is a care companion for older adults who live alone, and for the family who worry about them.

It learns the usual rhythm of a home from Ring events only: indoor motion detectors, door contact sensors and the doorbell. No video, no audio. From those it builds a per-home baseline of a few daily markers (when the morning starts, first kitchen activity, the front-door trip, night-time trips, the longest quiet spell) and explains any deviation in plain words, such as "No kitchen activity yet by 11:05 AM (usually 7:53 AM, give or take 20 min)."

When a day looks different, Alexa+ checks in with the resident first: "Good morning, Margaret. It's been a quiet morning so far, so I'm just checking in. How are you feeling today?" If she says she's fine, nobody else is told. If there is no answer after two gentle asks, or she says she fell or needs help, family gets an alert with the reasons. Family can ask their own Alexa "How is Mom doing today?", "How did she sleep?" or "Any alerts?", and see a card on screen devices.

The resident stays in charge: she chooses what family sees (essentials, daily rhythm or room-level detail), sets quiet hours, pauses it while away, asks "What does my family see?", reads an audit log of every family view and alert, and can erase her data.

How we built it

  • MCP server for Alexa+ on the official MCP Python SDK 2.3 (MCPServer), served over Streamable HTTP at /mcp. It negotiates protocol 2025-11-25 (the version the Alexa+ MCP Toolkit documents) and also the 2025-03-26 and 2025-06-18 handshakes shown in Alexa+ doc examples, plus the stateless 2026-07-28 revision. Twelve tools with Pydantic output schemas and annotations; four of them bind an MCP App card (ui://hearthbeat/care-card-<hash>.html) that renders on Echo Show and phone sizes through the MCP Apps postMessage protocol.
  • Ring integration built from the Ring API documentation: webhook v1.1 envelopes verified with HMAC-SHA256 over the raw body (X-Signature: sha256=...), idempotent on meta.request_id; motion_detected, button_press, contact_sensor_faulted/cleared, device_online/offline and device_added; a REST client for /v1/devices (JSON:API with included status and capabilities) and the Event History API. Because real Ring access needs identity verification, a device and a US Ring Protection plan, a local Ring sandbox serves the same shapes and sends signed webhooks to the real receiver. Every sandbox event is labelled simulated.
  • dayrhythm, a new dependency-free library: markers, robust baselines (median and MAD), and explanations with severity caps (a skipped morning walk is worth noting, never an alarm on its own).
  • Care loop: check-in, re-ask after 15 minutes, escalate with reasons; quiet hours; urgent night-door rule; offline sensors suppress their own markers; answers classified as ok / unwell / help / unclear.
  • Account linking that follows the Alexa+ Authentication and Account Linking pages: client_credentials service tokens (scope mcp:service), authorization code + PKCE S256 user tokens with rotating refresh tokens, RFC 9728 and RFC 8414 metadata, and 401 without WWW-Authenticate. Residents create a home on first link; family joins with the home's invite code.
  • Simulated Alexa+ host that opens real MCP sessions (2025-11-25) to the server, lets an open model on Nebius Token Factory (Qwen3-30B-A3B-Instruct) choose tools, falls back to a deterministic router with no key, renders the MCP App card in a sandboxed iframe, and speaks with the Web Speech API.
  • Web app (React 18, Vite, Tailwind v4, Motion): landing page whose hero is the live product, caregiver app (today, rhythm charts, alerts, privacy and audit log, Ring devices) and the two-device Alexa conversation view.
  • Quality: 64 automated tests (57 server, 7 library), including raw JSON-RPC checks in the style of MCP Inspector; validated with the official MCP Inspector CLI; warm get_wellbeing_status p50 14 ms on loopback.

Challenges we ran into

  • The Alexa+ add-ons console shows "Coming Soon" for our developer account, and the Alexa AI CLI and Local Inspector packages were not on public npm, so we could not deploy to the Alexa+ web simulator. We built the documented simulated-Alexa+ path and kept the server strictly to the Alexa+ rules so it can onboard unchanged.
  • The Alexa+ auth requirements (client credentials, no WWW-Authenticate) do not match the MCP SDK's built-in auth, so we wrote a small OAuth server.
  • Docs disagree on the protocol version Alexa+ sends (2025-11-25 vs 2025-03-26 vs 2025-06-18); we support and test all.
  • There is no documented way for an MCP add-on to make Alexa speak first, which care-taking needs.
  • Ring contact-sensor payloads are listed but not shown; we assumed the base envelope and made sensors' rooms a household setting, since Ring payloads don't say where a sensor is.

Accomplishments we're proud of

  • A check-in that de-escalates: in the demo, "I'm fine, I just slept in" ends it and family is never bothered.
  • Every alert carries its reasons; every family view is in the resident's audit log.
  • Zero check-ins in the offline-sensor scenario: a dead battery is not an emergency.
  • An honest, reusable open-source baseline library.

What we learned

  • Explainability and consent are product features, not compliance chores: the same sentence that justifies an alert is what lets the resident trust the system.
  • MCP's 2025-11-25 Streamable HTTP plus MCP Apps is enough to build a voice-plus-screen experience that runs in any compliant host.

What's next

  • Onboard to Alexa+ (development stage and web simulator) as soon as the add-ons console is available to us; link a Ring Playground token to list real sandbox devices; pilot with a few US households who already own Ring Alarm.
  • Proactive check-ins through a native Alexa+ capability, if one is offered to MCP add-ons.
  • Weekday/weekend baselines and seasonal drift in dayrhythm; per-household tuning that the resident can see.

Links

Ring events and the Alexa+ host are simulated in the demo and labelled on screen; the MCP server (spec 2025-11-25, Streamable HTTP), the routine engine and the apps are real. Built solo by Yashvardhan Thanvi with Claude Code as an AI pair programmer; the narration in the demo video is a synthetic voice.

Built With

  • alexa
  • claude-code
  • docker
  • mcp-apps
  • mcp-python-sdk
  • motion
  • nebius-token-factory
  • oauth2
  • pkce
  • pydantic
  • pytest
  • python
  • qwen
  • react
  • ring-api
  • sqlite
  • starlette
  • streamable-http
  • tailwindcss
  • typescript
  • uvicorn
  • vite
  • web-speech-api
Share this project:

Updates

Submission history