Inspiration

If you collapse at your desk or in your room, the one device that could call for help, your phone, is often out of reach, and you may not be able to use it anyway. Most fall-alert products are wearables that have to be charged and worn. We asked a different question: what if the space itself could notice, ask "Are you okay?", and bring in someone you trust if you can't answer?

What it does

GuardianLens watches one consenting adult through one fixed camera and looks for a possible collapse. It never claims to diagnose anything.

Detect. Local body-pose estimation feeds experimental movement rules. Seated mode (desk and laptop) needs only head and shoulders, calibrated to the person's normal posture.
Ask. A spoken and on-screen "Are you okay?" gives 10 seconds to respond. With no answer, "Do you need help?" starts a second countdown. "I'm okay" cancels at either stage, on the laptop .
Call trusted contacts. If nobody responds, GuardianLens works down a contact list the user wrote in advance, calling only contacts who opted in. A contact must press 1 to acknowledge. Picking up is never treated as acknowledgment, and wrong digits, voicemail, declined calls and failures all move on to the next contact.
Guide a bystander. Anyone nearby can tap "I'm here to help" to see what the camera observed, the location captured at detection, that the cause is unknown, first-aid guidance, and a reminder to call their local emergency number (911 in the US) right away without waiting for the contact plan. This reduces panic and gets the right information to the right person.
Never overclaim. Answered, acknowledged and resolved are separate states. Acknowledgment does not mean help has arrived. The final emergency-service step is simulated: actual emergency dialing is disabled.

How we built it

Detection: MediaPipe Pose Landmarker running locally in the browser/Electron renderer (WebAssembly, CPU). Video never leaves the machine and no raw video is stored. A seated detector compares the calibrated head-to-shoulder ratio, normalized by shoulder width, against live frames, with multi-frame confirmation so one noisy frame cannot start an incident. A full-body mode is also available.
Response engine: a pure, timer-driven TypeScript state machine (check-in stages, contact plan, call outcomes, escalation), kept separate from I/O so every path could be unit-tested.
Server: Node.js with WebSockets keeps the dashboard and a paired iPhone in sync. Owner and phone access use separate tokens, and the server validates every command and every saved file.
Phone calls: the server places calls through the Twilio Voice API with plain fetch. Twilio's webhooks are checked with HMAC-SHA1 signature verification using only node:crypto, so no new dependency was needed.
Desktop and UI: React + Vite dashboard, an Electron shell with a menu-bar icon, and browser text-to-speech for the spoken check-ins.
Python detector: a standalone MediaPipe/OpenCV version with the same rules, so detection can also run outside the browser.
Testing: 167 automated tests cover the state machine, detector rules, security checks, call lifecycle, cancellation and the keypad flow.

Challenges we ran into

Not crying wolf. Leaning in to read or bowing looks like a collapse to a pose model. Tuning thresholds, confirmation frames and calibration was a constant trade-off between missed events and false alerts, and we have not yet measured those rates formally.
Honest state. It is tempting to treat an answered call as "someone's coming." Keeping answered, acknowledged and resolved separate, and keeping "simulated" labels truthful everywhere, took deliberate design.
Privacy by default. Camera consent defaults to off, processing stays local, and phone access uses short-lived pairing links over an HTTPS tunnel.
Real telephony. Webhooks have to be authenticated, calls cancelled when the person answers "I'm okay," and stale callbacks ignored. We tested this against a simulated provider.

Accomplishments that we're proud of

A complete, testable path from possible collapse to a person who explicitly acknowledged, with every failure branch handled.
A design that is upfront about what it cannot know.
Signature-verified call webhooks with no extra dependency.

What we learned

Safety software is mostly about its failure paths and its honest wording. Making the engine pure and testable let us change detection and calling without breaking the response logic.

What's next

Measure missed events, false alerts and response time on safe staged recordings and ordinary activity.
Have the laptop speak to bystanders aloud ("A person may need help. Please call 911.") and automatically notify nearby people.
Real-device testing: native Mac tray behavior, iPhone over the HTTPS tunnel, and push notifications.
Speech responses ("I'm fine") and an inactivity detector.
Clinical and accessibility review. GuardianLens is a hackathon prototype, not a medical device, and makes no diagnostic or clinical-accuracy claim.