-
-
Simulated Alexa+ style assistant. Each answer comes from real calls to our MCP server; the raw tool result is shown under the chat.
-
Simulated figure and timing: our alert at 5.7 s, a plain fall detector stand-in says FALLING at 7.4 s. Not a measurement of any product.
-
In short: a family member asks Alexa+ "How is mom this morning?" and hears a few plain sentences built from what a camera node at home noted: she stood up, she was unsteady for about three seconds, she has been still for an hour. Concept demo on simulated data: a self-hosted MCP server with four tools, a simulated Alexa+ experience and 18 tests.
Inspiration
Fall sensors and panic buttons work only after something has happened, or when the person presses the button. For a family that lives far away, the useful question comes earlier: is she steady this morning? We wanted that answer to come from the assistant they already talk to, in a few plain sentences, and without the person at home having to do anything.
What it does
A camera node at home keeps video on the device and runs a small pose model. A rule layer turns the pose into short event lines: stood up, unsteady for about three seconds, long stillness, lying down. Only those lines are stored.
Our self-hosted MCP server (spec 2025-11-25, Streamable HTTP) exposes four tools: get_person_status, get_recent_alerts, get_daily_summary and acknowledge_alert. A daughter asks "How is mom this morning?" and hears: "Mom is sitting. She last moved 62 minutes ago. This morning she stood up twice and was unsteady once, for about 3 seconds. A note went to the care contact. That is only what the camera sees."
The server remembers, per caller, what was already told, so "Did anything happen since I last asked?" works across conversations. "I will call her now" marks the alert as seen. In the simulated assistant a voice policy rejects anything that diagnoses, predicts, scores or reassures, and the answer says so when the camera has gone quiet. The tools themselves return only plain facts and a short note.
Apart from one public comparison clip in the video, everything in the demo is simulated. The pose stream comes from a stick-figure simulator, the Alexa+ experience is a simulated web assistant that makes real MCP calls for every answer, and the note to the care contact is not sent anywhere. A second view compares the timing with a usual fall detector. In the repository it runs on simulated stick-figure frames and the timing comes out of the rule code. In the demo video the same view plays a real public clip (AiBrainPulse), slowed down: the usual detector labels follow the clip, while our alert time is simulated.
How we built it
Python 3.12, the MCP Python SDK 2.3 (MCPServer with streamable_http_app), Starlette, SQLite. The rule layer works on MediaPipe-style landmarks (trunk lean, hip sway with the walking trend removed, posture from leg to torso ratio). A voice policy module in the simulated assistant builds the spoken answer (60 words at most) and rejects diagnosis, prediction, scores and reassurance. 18 pytest tests cover the rule layer on simulated scenarios (including a control day that must give no unsteady event), the voice policy, and the MCP server over real Streamable HTTP (handshake at 2025-11-25, tools, state across sessions, acknowledgement). The demo video is recorded by a script from the running app.
Challenges we ran into
Keeping a voice answer short enough and plain enough, and deciding what the assistant must never say. Walking first looked like swaying until we removed the walking trend from hip movement. The 2.x SDK client negotiates a newer protocol version by default than the track minimum, so the demo client is pinned to the 2025-11-25 handshake and a test checks it. The Alexa+ docs for a real add-on (OAuth 2.1 with PKCE, a public HTTPS endpoint, the Alexa AI CLI) are separate from the rules page. We found them late and did not build that layer in the time window, so the Alexa+ side is the simulated path the rules allow.
Accomplishments that we're proud of
A complete loop on simulated data: pose, event, MCP tool, short spoken answer, acknowledgement, with state kept between conversations, tests, and a video that is recorded from a script. The usual-detector comparison computes its timing from the actual rule code instead of a hard-coded number.
What we learned
Caretaking needs answers, not dashboards. Voice length is a hard budget. The line between what the camera saw and what it means has to be enforced in code, not left to wording.
What's next
- Add OAuth 2.1 with PKCE, deploy the server as a real Alexa+ add-on and test it in the Alexa+ simulator.
- Run the live camera adapter on real, consented recordings before saying anything about real-world behaviour.
- A Ring path for adult children who live far away.
- Independent evaluation before any performance claim.
About the team
Built by GV66 CO. Our technology is patented, so we do not go into it here. This project is a concept demo built from public components and our own code, on simulated data.
Log in or sign up for Devpost to join the conversation.