Guardian

An AI Voice agent for Elderly Integrated with Alexa+

Comment

Inspiration

For many older adults living alone, the front door is the biggest daily worry. Is that a delivery or a stranger? Should I open it? A knock at the wrong time can be frightening, and the people who care about them, family or a hired carer, often can't be there. Doorbell apps help, but they assume you can pick up a phone, find an app and interpret a video clip. We wanted something an elderly person can simply ask out loud, with a caregiver one sentence away.

What it does

Guardian is a voice-first front-door companion for Alexa+, built as a self-hosted MCP server.

  • Ask, don't check. "Alexa, who's at the door?" Guardian looks at the front-door camera and answers in plain spoken language, such as "A delivery person is leaving a package on the porch."
  • Offers help when it matters. If the visitor looks unfamiliar or unclear, Alexa asks: "Would you like me to send a message to your caregiver?" One "yes" sends it.
  • Notices when home goes quiet. Guardian learns the household's normal door activity hour by hour. If a normally busy period is unusually quiet, it can flag that to the caregiver too. Quiet hours (like overnight) are respected, so it doesn't raise false alarms.
  • Reaches the right person. The caregiver can be a family member or an appointed carer. They receive a real push notification, with no app account required.

How I built it

  • MCP server (TypeScript): a Streamable HTTP server exposing six tools: get_latest_doorbell_event, describe_door_snapshot, list_recent_door_events, check_household_activity_pattern, notify_caregiver, and guardian_status. It runs statelessly, with a fresh server and transport per request.
  • Agent gateway: a genuine MCP client that discovers the tools at runtime and runs a Gemini tool-calling loop, with retries and model fallbacks for the free tier.
  • Vision: Gemini describes the scene and classifies the visit as delivery, visitor, vehicle or unclear, with a suggested action.
  • Anomaly detection: a per-hour baseline of door activity, with thresholds separating "quiet" from "unusually quiet".
  • Notifications: ntfy.sh push notifications to the caregiver.
  • Voice: a simulated Alexa+ device in the browser, using the Web Speech API for both speaking and listening.
  • Open source: MIT licensed, with a README, demo script and friction log.

Challenges I ran into

  • Ring access. Ring has no open public API, and its official Partner API needs a verified developer registration. The Ring integration is built behind a switch, but the demo runs on a realistic simulated door-event generator because i had no camera. This is stated openly in the README.
  • Model churn on the free tier. Planned Gemini models were retired for new users partway through, and busy periods returned 503 errors. I added a fallback chain so the agent keeps answering.
  • Thought signatures. Gemini 3 tool calls must have their reasoning signatures replayed exactly, or the next request fails with a 400. Fixing it meant changing how the agent loop stores each turn.
  • Honesty with vulnerable users. For an elderly person, a wrong confident answer is worse than "I'm not sure." Guardian never invents a description of footage it couldn't analyze, and tells the user when the camera couldn't be read.
  • Browser voice quirks. Chrome loads voices asynchronously and cuts off long speech, so replies are spoken sentence by sentence.

What we learned

MCP draws a clean line between the assistant and what it can do. Guardian's six tools can be audited at a glance, and any MCP-capable agent can use them. We also learned that designing for older users changes the priorities: short answers, a single yes/no question, a clear spoken confirmation, and never overstating what the camera saw.

What's next

  • Connect a real Ring camera through the live integration.
  • Replace the simulated device with a native Alexa+ skill.
  • Let the caregiver reply by voice message, and add escalation rules if the first caregiver doesn't respond.

Built With

Share this project:

Updates

Submission history