Vigilant

Logo

Inspiration

Many older adults want to keep living independently, but their families and caregivers worry constantly about what might happen when no one is around. A fall can go unnoticed for hours, and the longer someone lies on the floor without help, the more serious the consequences become. Falls aren’t the only risk, either. Poor nutrition, a room that’s too hot or too humid, or a slow change in daily routine can quietly affect an elder’s health. Wearables like panic buttons only work if the person is conscious, willing to wear them, and able to press them. We wanted something that works passively: a system that watches over a person and their environment without requiring them to do anything, and tells the right people the moment something is wrong.

What it does

Vigilant is a monitoring system that uses a camera and environmental sensors to detect movement, track wellbeing, and alert caregivers and family when an elderly person may need help. It:

Detects falls by analyzing body position and movement in real time Tracks presence, determining whether the person is in frame or out of frame, and for how long

Monitors camera uptime, logging how long the camera has been on so caregivers know the system is working

Records environmental conditions, including temperature and humidity, to flag unsafe or uncomfortable surroundings

Logs food intake and nutrients, capturing what the person ate and the nutritional content (calories, protein, vitamins, and so on) so caregivers can spot poor eating habits

Stores everything in a database with a timestamp, location (which room or home), and the device that captured it, so every event is traceable and patterns can be reviewed over time

Sends notifications to caregivers and family members when something concerning happens, such as a detected fall, prolonged absence from the frame, dangerous room conditions, or missed meals

How we built itWe built Vigilant around a computer vision pipeline that processes the camera feed frame by frame. Pose and movement detection identifies the person’s body position, which lets us distinguish normal activity from a fall (for example, a rapid change from upright to horizontal followed by no movement). The same pipeline determines whether the person is in or out of frame and tracks how long they have been in each state.

Alongside the camera, sensors report temperature and humidity readings. Food intake is logged and matched against a nutrition database to calculate the nutrients in each meal.

Every event, whether a fall, presence change, sensor reading, or meal, is written to a database with its time, location, and source device. A notification service watches for critical events and thresholds (a fall, a long absence, extreme temperature, or no food logged all day) and sends alerts to a list of caregivers and family members. A dashboard lets caregivers view current status, history, and trends.

Challenges we ran into

Reducing false positives: Sitting down quickly, bending over, or picking something up can look like a fall. We tuned detection thresholds and added checks like “no movement after the change in posture” to avoid alarming families unnecessarily.

Real-time performance: Processing video fast enough to react within seconds, without powerful hardware, required optimizing the pipeline. Lighting and camera angles: Detection accuracy varied with low light, occlusion by furniture, and camera placement.

Defining “out of frame”: Someone leaving the room is normal, but being out of frame too long could signal a problem, so we had to set sensible time limits.

Combining multiple data sources: Camera events, sensor readings, and food logs all arrive in different formats and at different rates, so designing one database schema that ties them together by time, location, and device took careful planning.

Accurate food and nutrient tracking: Estimating what someone ate and its nutritional value is difficult, especially with portion sizes.

Reliable notifications: Making sure alerts reached the right people quickly without flooding them with messages.

Privacy: Putting a camera in someone’s home is sensitive, so we thought carefully about how data is handled and stored.

Accomplishments that we're proud of

Building an end-to-end system, from sensors and camera input to detection, database logging, and caregiver notifications

Detecting falls and presence changes in real time

Going beyond emergencies to capture a full picture of wellbeing, including environment and nutrition

Creating a historical record that gives caregivers insight into daily patterns and trends

Building something with real human impact, designed to give families peace of mind and help seniors stay independent longer

What we learned

We learned that motion detection alone can’t detect falls, since you need posture and speed together, plus debouncing to avoid false alarms. We found that Gemini was great for turning messy TigerData time-series records into a clean, usable dataset, and that on-device processing let us keep the camera feed private. Most of all, we learned to be upfront about our limits, which pointed us to our next steps.

What's next for Vigilant

Improved accuracy with better models and testing across more environments, lighting conditions, and body types

Privacy-first processing, such as running detection on-device and storing only skeleton data or events rather than raw video

Multi-room and multi-camera support to cover an entire home Automatic food recognition, using the camera to identify meals and estimate nutrients without manual logging

A mobile app for caregivers and families with live status and instant push alerts

Health insights, like detecting changes in routine, reduced mobility, poor nutrition, or unusual inactivity that might signal declining health

Smarter environmental alerts, such as warning when heat or humidity reaches levels risky for seniors

Escalation rules, so if the first contact doesn’t respond, the system alerts the next person or emergency services

Integration with smart home devices and wearables for a fuller picture of wellbeing