GuardSight

AI-powered that helps flag caregiver abuse, notifing family of vulnerable individuals who are unable to report the abuse themselves.

Comment

GuardSight

Inspiration

Caregiver abuse is one of the most underreported forms of harm, with studies estimating that nearly 90% of cases go unreported. It has been frequently shown on the news of caregiver abuse cases coming to light months to years after the original event occurred. Many victims—particularly elderly individuals, cognitively impaired patients, and dependent adults—are often unable to advocate for themselves. As a result, harmful interactions frequently occur without witnesses or documentation.

This issue is compounded by the environments in which care takes place. Whether in private homes, assisted living facilities, or care centers, many interactions happen behind closed doors. When concerns arise later, there is often no objective record of what occurred.

We set out to address this gap by asking a simple question:

What if there was always a witness?

GuardSight was built to act as an intelligent observer—one that does not replace human judgment, but ensures that potentially concerning interactions are less likely to go unnoticed.

What It Does

GuardSight is an AI-powered safety monitoring platform that analyzes video footage to identify interactions that may warrant further review.

The system functions as an early-warning tool, surfacing events and providing context so trusted individuals can make informed decisions, and ensure safety + peace of mind.

Core Features

  • Dual-role accounts

    • Patient (monitored individual or representative)
    • Family member (receives alerts and reviews incidents)

  • Real-time video monitoring

    • Browser-based recording using the Camera API
    • Near real-time frame analysis during recording

  • Retrospective video analysis

    • Upload and analyze previously recorded footage

  • AI-assisted classification

    • Interactions are categorized to prioritize review:
Classification Description
normal_caregiving_assistance Expected, non-concerning interaction
accidental_movement Unintentional contact or movement
aggressive_handling Unusually strong or abrupt handling
potential_physical_abuse Potentially concerning, requires attention
unknown Ambiguous, needs additional context

  • Automated alerts

    • Email notifications for high-risk events
    • Includes:
    • Confidence estimate
    • Explanation of why the event was flagged
    • Before/during/after frames
    • Timestamp and metadata

  • Family dashboard

    • Chronological, filterable history of flagged incidents

  • Persistent incident storage

    • Secure PostgreSQL database for long-term tracking and review

Human Review First

GuardSight is designed as an assistive system, not an autonomous authority.

It does not:

  • Determine whether abuse occurred
  • Assign blame
  • Make legal conclusions

Instead, it ensures that important interactions are surfaced with enough context for human evaluation.

How We Built It

Architecture

GuardSight is a three-tier distributed system:

Layer Technology Stack Purpose
Frontend React + Vite UI, video capture, dashboards
Backend API Node.js, Express Auth, routing, alerts, data handling
AI/CV Service Python, FastAPI Pose detection, frame analysis
Database PostgreSQL (Neon) Users, incidents, relationships

AI Pipeline

  • Pose Detection: YOLOv8 (Ultralytics) extracts keypoints from video frames
  • Frame Processing: OpenCV handles preprocessing
  • Contextual Analysis: Google Gemini evaluates interactions and generates classifications

Pipeline Flow: Video → Pose Detection → Event Trigger → AI Classification → Alert → Human Review

Key Design Decisions

  • Asynchronous processing

    • Events are handled via background jobs to reduce latency

  • Context-aware prompting

    • Medical conditions are included to reduce false positives

  • Multi-person detection requirement

    • Events are only flagged when multiple individuals are present

Challenges We Ran Into

1. Detection Sensitivity vs. False Alerts

Early versions flagged nearly all motion as concerning, including routine caregiving actions.

Solutions:

  • Require multiple people in frame
  • Incorporate patient medical context into AI prompts
  • Use conservative escalation to avoid over-alerting

2. Latency in AI Processing

Initial synchronous processing caused 5–10 second delays per event.

Solution:

  • Implemented asynchronous job system:

    • Immediate job creation
    • Non-blocking processing
    • Polling endpoint for results

This enabled real-time responsiveness and concurrent event handling.

Accomplishments

  • Fully functional end-to-end system

    • Authentication, live monitoring, alerts, and historical review all operational

  • Complete AI-assisted monitoring pipeline

    • Automated detection, classification, and alerting workflow

  • First computer vision deployment

    • Successfully implemented pose estimation and multi-person tracking

  • Real-world impact focus

    • Addresses a meaningful and underexplored safety problem

What We Learned

  • Balancing sensitivity and specificity is critical

    • Too many alerts reduce trust; too few reduce usefulness

  • Asynchronous architectures are essential for AI systems

    • Long-running tasks must not block real-time interaction

  • Context improves AI reliability

    • Incorporating domain-specific information (e.g., medical conditions) significantly improves results

  • Computer vision pipelines require iteration

    • Model selection, thresholds, and tuning all impact performance

What’s Next

Short Term

  • Complete and refine the video upload analysis workflow

  • Enable export of incident packages:

    • Frame captures
    • Event summaries
    • Downloadable reports

Medium Term

  • Privacy-preserving AI

    • Replace external APIs with self-hosted models
    • Enable HIPAA-compliant deployments
    • Relocate incident report data from Neon to a secure database with enhanced RLS

  • Expanded detection signals

    • Voice and distress detection
    • Fall detection
    • Additional body movement tracking
    • Facial expression analysis

  • Mobile applications

    • iOS and Android apps for passive monitoring

Long Term

  • Dedicated smart camera hardware

    • Motorized tracking
    • 360-degree coverage
    • On-device AI processing

  • Care facility integration

    • Nursing home systems
    • Compliance platforms
    • Incident reporting tools

  • Scalable deployment

    • From individual households to institutional environments

In conclusion, GuardSight represents a step toward increasing visibility, accountability, and safety in environments where vulnerable individuals need it most. By ensuring that critical moments are captured and surfaced, the system empowers humans—not replaces them—to make better, more informed decisions.

Built With

Share this project:

Updates