Output
AWS alert
AWS alert

Inspiration

Large public gatherings, rallies, festivals, transport hubs, and emergency situations can become unsafe when crowd density increases faster than security teams can respond.

Traditional CCTV monitoring requires operators to continuously watch multiple screens. Basic object detectors also struggle with dense crowds because people overlap, appear at different scales, and may be viewed from moving drones or elevated cameras.

We wanted to build a system that does more than count people. CrowdSafe Vision converts visual crowd behavior into practical safety intelligence: how many people are present, where they are concentrated, whether movement is abnormal, and when an operator should be alerted.

What it does

CrowdSafe Vision is an AI-powered crowd monitoring platform for images and drone-view videos.

It provides:

Point-based human-head localization for dense crowds
Crowd counting and zone population estimation
Unique-subject tracking across video frames
Capacity-limit monitoring
Chaos and rapid-movement anomaly detection
Drone-motion compensation
Polygon-based zone fencing
Density heatmaps and visual overlays
Real-time WebSocket telemetry
Session history and downloadable reports
AWS CloudWatch monitoring
AWS SNS safety notifications

The system is designed for event security teams, public-safety operators, smart-city command centers, and organizations managing large venues.

How we built it

CrowdSafe Vision uses a React/Vite dashboard connected to an asynchronous FastAPI backend.

Computer vision pipeline

The backend uses OpenCV for:

Image and video decoding
Color-space conversion
CLAHE contrast enhancement
Optical-flow feature tracking
Affine camera-motion estimation
Polygon zone filtering
Heatmap rendering
Motion-vector overlays
Annotated frame generation
JPEG encoding

For crowd localization, we use a point-based P2PNet model with a VGG-16 backbone. Instead of drawing large bounding boxes, the model predicts points near individual heads. This is better suited to dense scenes where bounding boxes overlap heavily.

The inference engine supports image magnification, confidence thresholds, tiled processing, overlapping-patch merging, duplicate suppression, CPU execution, and CUDA acceleration when available.

Tracking and movement analysis

For videos, detected head points are passed to a tracking pipeline that combines:

Lucas–Kanade optical flow
Global affine motion estimation
Drone-drift compensation
Euclidean-distance matching
Hungarian assignment using SciPy
Track aging and removal
Velocity estimation
Cumulative unique-subject counting

This helps distinguish actual crowd movement from movement caused by a moving camera.

Safety and alert pipeline

The system evaluates crowd counts and movement behavior against configurable thresholds.

It can detect:

Normal crowd conditions
Elevated density
Capacity-limit breaches
Rapid movement by multiple tracked subjects
Possible chaos, stampede-like movement, or counterflow
Crowd concentration inside a selected zone
Predicted capacity breaches based on recent count trends

The frontend displays SAFE, MODERATE, and DANGER threat states.

AWS integration

CrowdSafe Vision sends operational telemetry to AWS CloudWatch, including crowd-size and anomaly metrics. This allows deployments to be monitored through cloud dashboards and alarms.

We also use AWS SNS to notify subscribed operators when important crowd-safety events occur.

The complete decision flow is:

Video or image input ↓ OpenCV preprocessing ↓ P2PNet head-point detection ↓ Tracking and crowd analysis ↓ Capacity or movement-risk evaluation ↓ AWS CloudWatch telemetry ↓ AWS SNS notification ↓ Human operator response