Station Crowd Safety Agent

• 

  Live Dashboard

Inspiration

Crowd crushes build up in minutes, usually at the places stations already know are risky: stairs, footbridges, gate queues and platform edges. A control room can't watch every camera at once, and by the time someone notices, the crowd is already dangerous. We wanted a system that watches continuously, tells an operator where a crowd is becoming unsafe and why, and then leaves the decision to a human.

What it does

The Station Crowd Safety Agent turns station video into a live, explainable risk picture for each zone (stairs, platform edge, gate, concourse):

• Measures people per square metre and crowd turbulence in each marked zone.
• Decides a risk level (Normal, Watch, Warning, Critical, or Can't trust) using transparent rules, not a black box.
• Recommends an action for the zone type, such as holding entry at an upstream gate.
• Keeps a human in charge. Warning and Critical alerts wait for an operator to approve or dismiss them on the dashboard.
• Never says "all clear" blindly. A low-confidence reading or a failed camera is reported as Can't trust, and a person is asked to check.

How we built it

• OpenCV 5 does the video and image work: reading and writing video, dense optical flow with calcOpticalFlowFarneback, zone masks with fillPoly and pointPolygonTest, blurring the head region of every detected person with GaussianBlur, and quality checks such as brightness and blur with cvtColor and Laplacian.
• YOLO11n (through Ultralytics) detects people. Each person is assigned to a zone by the point where their feet meet the ground.
• Density and turbulence. For a zone with $N$ people and floor area $A$, density is $\rho = N / A$ people/m². Turbulence compares the average flow vector with the average flow magnitude:

$$\text{turbulence} = 1 - \frac{\left\lVert \overline{\vec{v}} \right\rVert}{\overline{\lVert \vec{v} \rVert}}$$

It is near 0 when everyone moves the same way and near 1 when movement is chaotic.

• Risk rules. Density at or above 4 people/m² is Warning, and at or above 5 is Critical. High turbulence with density at or above 4 is also Critical. A linear regression over a 60-second window gives the density trend, and hysteresis stops a zone on a threshold from flapping between levels.
• Safety by design. The rule engine alone sets the level. An optional Gemini layer can only add a plain-English summary, and it can never change a level or a recommendation.
• Dashboard. FastAPI streams alerts to the browser with server-sent events, shows the annotated video in step with the zone cards, and handles approve and dismiss.
• AWS bridge. Evidence frames can go to Amazon S3, and Critical alerts to Amazon SNS. Without credentials it runs in a simulation mode that only logs what it would have sent.

Challenges we ran into

• Detection undercounts dense crowds. On our platform footage the detector clearly misses much of the far, packed crowd. So the density we measure there is a lower bound, and the real footage never reaches Warning. We state this openly instead of hiding it.
• Scripted scenarios for the alerts. Because the real clips are low-density, we use scripted readings (a 10-minute build-up) to show Watch, Warning and Critical. We label them as simulated.
• Handheld footage. Camera motion looked like crowd turbulence, so we subtract the global flow before measuring it. The result is still noisy.
• Alert storms. Critical alerts repeat every second, which would flood the operator and send an SMS every second through SNS. We added hysteresis and a per-zone cooldown.
• Speed. On a laptop CPU the pipeline runs at about 2 to 6 frames per second, so it works offline and the dashboard replays its results.

What we learned

• Rules and an LLM have different jobs. Rules decide, and the LLM explains.
• Being honest about what a vision system cannot see matters as much as the demo. We wrote down what we measured, and what we did not.
• Good safety tools need a "Can't trust" state.

Evaluation (what we actually measured)

• On the scripted scenarios, the normal run raised 0 alerts. In the build-up, the stairs reached Watch at 3:46, Warning at 7:45 and Critical at 9:49. A camera that went dark was flagged at 4:00.
• Throughput was about 2.0 fps on both clips, on a CPU-only laptop.
• We checked detections by eye on one frame per video.

What's next for Station Crowd Safety Agent

Real camera calibration to replace our estimated zone areas, person tracking to smooth the counts, detection of a moved or blocked camera from real video, a GPU or lighter detector for real-time speed, and an evaluation on labelled footage.

This is a prototype.

Built With

• amazon-sns
• javascript
• opencv
• python
• pytorch
• ultralytics
• yolo