Kinwise

Inspiration

Courier scams unfold in two stages: a frightening call from someone posing as a bank or government agency, then a "courier" who arrives later to collect cash or gold. Each device sees only one moment, so nothing connects the call to the knock at the door. Kinwise connects an earlier, explainable scam-warning signal to a later visitor event, then gives the resident a consent-controlled way to pause and seek help.

Older adults are not careless; they are targeted, often under pressure and alone. People aged 60 and over reported about USD 7.7 billion in fraud losses in 2025 (AARP, reporting on FBI data). In FBI Boston's courier-scam cases from 2023 to May 2025 (103 pickups, USD 26,024,691 lost), roughly 98% of the losses were reported by people over 60 (FBI Boston). We wanted to help Asha, a capable woman living alone, take a calm moment before an irreversible decision, without watching her or acting for her unless she chooses. Our idea is not another assistant, doorbell alert or notification: it is cross-event correlation combined with resident-controlled consent, privacy and honest delivery reporting.

What it does

1. Recognize a warning sign. In the labelled Alexa+ prototype, Asha asks, "Is this call real?" or saves a reminder. Fixed rules match documented scam-warning patterns and retain only the category of each matched sign, never her words.
2. Ask for consent. Kinwise offers a six-hour safety watch. Nothing starts and nobody is notified until Asha explicitly consents through a button on her authenticated TV. A spoken "turn it on" is refused because Alexa cannot reliably establish who is speaking.
3. Connect the events. In the demo, a clearly labelled simulated or signed door event stands in for the unavailable Ring person-at-the-door scenario. When a qualifying visitor event is processed during an active watch, the Vega app presents a captioned Pause explaining the earlier warning, with Call Meera available for Asha to request help.

Asha stays in control. "Not now" is the default focus. It keeps the watch off, discards the retained warning-sign categories (keeping only a note that she declined), and suppresses another prompt for one hour. If Asha does not respond, the consent question expires after 30 minutes. "Review privacy" explains what is retained without changing any settings.

Honest delivery reporting. The hub records a Pause as sent when it decides and queues it for the TV, shown only when the TV app reports rendering the screen, and not confirmed if no rendering report arrives within 60 seconds. "Shown" does not establish that Asha saw or understood the warning.

Watch the 2:49 demo: youtu.be/9CkonL1DmqE.

How we built it

• Consent in the backend, not a toggle. The TV carries a device-role token; MCP and voice tokens cannot activate the watch, and an Echo turn_on returns "needs the TV" and changes nothing. Tests prove it, and it matters because Alexa cannot tell who is speaking.
• Deterministic rules decide. The warning signs, the six-hour window and the door decision are rules with named constants. The model only talks kindly and describes a camera scene neutrally, so every Pause can be explained.
• Alexa+ track: a self-hosted MCP server (spec 2025-11-25, Streamable HTTP) with 11 role-scoped tools and four MCP Apps cards. The Alexa+ experience is a labelled simulation: an Echo Show web app that is a real MCP client and MCP Apps host.
• Fire TV track: a React Native for Vega OS app, D-pad first, captioned and VoiceView-friendly, run on the Vega Virtual Device.
• Ring track: the official Ring Partner API (live view, event history, signed webhooks) read through the Developer Playground by a worker that signs events to the hub.
• AWS Builder: Lambda and API Gateway, Cognito two-tier OAuth, DynamoDB, SNS e-mail, EventBridge Scheduler, Secrets Manager, Polly for the TV's voice, X-Ray, and a Strands concierge on Bedrock AgentCore with Amazon Nova 2 Lite, all in CDK.
• Open Source: alexa-plus-mcp-kit (Apache-2.0) and its alexa-mcp-conformance CLI.

See the architecture and consent boundary and the warning-to-Pause sequence.

Challenges we ran into

• Ring: the Developer Playground has no "person at the door" scene, so the door event in the demo is a labelled simulator or a signed event. The real worker reads the live feed and correctly stays quiet on an empty doorstep. We do not claim a genuine Ring person detection.
• Fire TV: a third-party app cannot draw over other apps, so the Pause appears only while Kinwise is on screen. We run on the Vega Virtual Device, not a physical Fire TV.
• Alexa+: participants cannot connect to live Alexa+, so the surface is simulated.
• Caregiver response: Meera's phone panel in the demo is simulated, and her response appearing on a physical television has not been verified. E-mail through Amazon SNS reached a real inbox in an earlier test (2026-10-14, documented in our README); we did not re-verify delivery for the final release, and SMS is AWS-sandbox only.
• A disclosed race: a notification already in flight may still be sent if she revokes consent at that instant. We bound it and say so instead of hiding it.
• Model wording: the concierge's spoken paraphrase can drift from the hub's exact text, so the hub is authoritative and the demo shows its words on the card.

Accomplishments that we're proud of

• Consent that is enforced, tested and explainable, with honest sent, shown and not-confirmed states.
• A scripted comparison against a door-only rule (evaluation report): Kinwise caught 2 of 3 scam scenarios and raised 0 false urgent alerts across 4 harmless visitors. An always-urgent door rule caught 3 of 3 but raised 2 false urgent alerts and sent 5 family notifications. Kinwise's one miss is a courier arriving after the six-hour watch expired. These are scripted scenarios, not a field study, and they do not show real-world effectiveness.
• 262 hub tests, 15 toolkit tests and 12 infrastructure tests pass; MCP conformance is 22/22 for the hub and 16/16 for the toolkit checker.
• After deployment we ran live checks against the hosted hub, including a 30-minute expiry run (deployment record).

What we learned

Consent and dignity are design constraints, not features: the resident owns every switch, family sees signals rather than recordings, and every look is logged where she can read it. The most trustworthy thing a safety product can do is say plainly what it sent, what is confirmed, and what it cannot know. Kinwise does not prevent fraud, identify anyone, or guarantee that a resident saw a warning.

What's next for Kinwise

SMS outside the AWS sandbox; a real Ring person event once the Playground offers one; testing on a physical Fire TV; and research with older adults and caregivers before anyone relies on it.

Built With

• alexa-plus
• amazon-nova
• api-gateway
• aws-cdk
• aws-lambda
• bedrock-agentcore
• cognito
• dynamodb
• eventbridge-scheduler
• fire-tv
• hono
• mcp
• mcp-apps
• polly
• python
• react
• react-native
• ring-api
• secrets-manager
• sns
• strands-agents
• typescript
• vega-os
• vitest
• x-ray