ElderWatch

• 

  Enrollment Setup

• 

  Bound Boxes

• 

  Live events

• 

  Data Insights

Inspiration

For many families, elderly loved ones tend to downplay what happened during the day. They may avoid speaking up about missed meals, reduced activity, or possible fall-risk moments because they do not want to worry anyone. We built ElderWatch to be a caring safety layer when family members are away at work or school and cannot be physically present at home.

What it does

ElderWatch is a real-time elderly monitoring and insights system powered by computer vision, structured event pipelines, and AI analysis.

• Behavior signal capture: Processes frame-level posture and activity signals from a live camera stream.
• Primary-subject association: Prioritizes accurate tracking of the elderly person so events map to the right individual.
• Structured event pipeline: Filters detections, applies confidence logic, and stores validated records in Snowflake.
• Timeline + insights dashboard: Presents recent events and trend summaries through a web interface.
• AI analysis chat: Uses weekly context (meals, falls, activity patterns) to answer caregiver questions in plain language.

How we built it

We implemented a distributed camera-to-cloud architecture designed for speed and observability.

• Capture + CV inference: Video frames are streamed to a CV service using YOLO8x.pt(w/Ultralytics) + OSNet_ain_x1_0 + MediaPipe (BlazePose heavy), that extracts pose, activity, and identity signals.
• Backend orchestration: FastAPI endpoints normalize, validate, and expose observation/event data to the frontend.
• Data layer: Snowflake stores raw observations, alerts, and derived event records for querying and trend aggregation.
• Insights generation: API routes compute daily trend metrics and timeline outputs for dashboard rendering.
• AI layer: Snowflake Cortex powers contextual chat responses grounded in the most recent week of structured data.
• Frontend: Next.js + React dashboard with polling, person filters, and demo-mode gating for safe mock-data demos.

Challenges we ran into

• Snowflake data populating + analysis quality: Early filtering was too harsh and suppressed useful behavior signals. We had to rebalance confidence thresholds to keep precision without losing meaningful context.
• Latency pressure on remote GPU path: Running on a RunPod 5090 (without local high-end GPUs) introduced extra network and processing latency that required careful tuning.
• Raspberry Pi 4 deployment fall-through: Our original hardware path depended on a Pi-based connector to the CV server/RunPod flow, but the RPi4 broke and forced deployment-plan changes mid-build.

Accomplishments that we're proud of

One of our biggest wins was achieving a sub-3-second end-to-end pipeline:

• webcam frame captured,
• frame sent to CV service,
• model returns frame-level data,
• outputs filtered by confidence/weight logic,
• records deposited into Snowflake,
• dashboard receives updates through consistent API polling.

Seeing this pipeline work live in sequence was a major milestone and validated both our architecture and optimization work.

What we learned

The hardest part of elderly monitoring with computer vision is behavioral fidelity, not just model speed. Building trustworthy monitoring requires accurately recreating real elderly movement patterns, routines, and day-to-day actions so the resulting insights are dependable for caregivers.

What's next for ElderWatch

Our next step is to shrink and harden the deployment footprint so ElderWatch can run on compact, portable hardware like an RPi4-class setup for real-world installation. The long-term goal is a practical, low-friction product families can deploy at home.

Built With

• fastapi
• mediapipe
• next.js
• osnet
• python
• react
• recharts
• runpod
• snowflake
• sql
• ts
• yolov8