Heads Up

Responder View
Dashboard View
Map View Showing Location of Alert

What it does

Heads Up turns existing security cameras into a proactive safety network for people who may be unable to call for help.

Designed for TTC stations, Heads Up detects potential falls and collapses in real time. When someone falls and remains down, the system alerts station security with a clear, plain-language explanation of what triggered the alert. Simultaneously, an automated announcement prompts nearby passengers to check on the individual. If the alert goes unacknowledged, it automatically escalates to a backup responder to help ensure no potential emergency is overlooked.

Safety shouldn't come at the cost of privacy. Heads Up processes video locally, displays only skeletal outlines to responders, and avoids exposing faces. Every alert is treated as a welfare check, prioritizing assistance, dignity, and care over enforcement.

How we built it

Computer Vision: Python, OpenCV, YOLO Pose, and ByteTrack detect body keypoints, track individuals, and identify sudden falls or gradual collapses followed by a sustained lying posture.
Backend: A FastAPI server manages incidents in SQLite and delivers real-time updates through WebSockets.
Responder Dashboard: A React dashboard enables responders to acknowledge, escalate, dismiss, and resolve alerts, with a complete audit log of actions.
Incident Mapping: Leaflet maps detected incidents to configured TTC camera locations, helping responders identify where assistance is needed.
Secure Incident Sharing: Expiring, revocable links provide authorized responders with read-only incident briefings.

Our prototype also includes simulated train tracking and an emergency-services workflow. No calls are placed to 911 automatically.

Challenges we ran into

Distinguishing falls from everyday movement: Sitting quickly, bending, or kneeling can resemble a fall. We tuned detection thresholds and timing logic to reduce false alarms.
Working with real-world camera angles: High-mounted, downward-facing cameras distort body proportions, making posture and torso-angle estimation more difficult.
Maintaining consistent person tracking: We worked to preserve each individual's tracking ID through a fall, even when other people temporarily obstruct the camera's view.
Testing safely: Rather than attempting uncontrolled falls, we used public fall-detection datasets and performed slow, controlled collapses onto a mat.

Accomplishments we're proud of

Built an end-to-end safety pipeline: Connected computer vision, real-time incident detection, live alerts, and responder actions in one working prototype.
Made alerts explainable: Each incident includes a plain-language explanation of why it was flagged, helping responders make informed decisions.
Prioritized privacy by design: Built the system around skeletal representations rather than identifiable facial imagery.
Implemented automatic escalation: Created a workflow that escalates unacknowledged alerts to a backup responder.

What we learned

Building and tuning real-time computer vision systems for challenging camera angles and movement patterns.
Integrating computer vision, backend services, and a frontend dashboard using live WebSocket communication.
Designing safety-critical technology requires more than detection accuracy. Privacy, dignity, and responsible escalation must be built in from the start.
Coordinating development across a team and integrating multiple components under a tight hackathon deadline.

What's next for Heads Up

Pilot with the TTC: Explore integration with existing station camera infrastructure, minimizing the need for additional hardware.
Expand to other environments: Adapt Heads Up for university campuses, hospitals, care homes, and parking garages.
Add AI-assisted verification: Introduce a second-stage vision model to assess detected incidents and help reduce false alarms before escalation.
Stress-test in real-world conditions: Evaluate performance in crowded environments, low-light conditions, and scenarios involving mobility aids such as wheelchairs.