Breathing Room: crowd pushing and minutes to danger

A crowd-crush early warning that reads crowd density and pressure in people from ordinary CCTV, predicts the next minutes with the physics of crowd flow, and gives a deadline before anyone falls.

Comment

Inspiration

At Itaewon, Seoul, on 29 October 2022, the first of 11 emergency calls about the crowd came at 18:34. Police investigators later estimated, from security footage, about 8 people per m² in the alley at 22:15 and 9 to 11 per m² by 22:25. 159 people died. Cities now count heads on CCTV. A count says how full a street is. It cannot tell a calm, packed crowd from one where people have started to push, and it cannot say how much time is left. Breathing Room measures both.

What it does

For every zone it measures three numbers from video: density in people per m², a pushing index from crowd motion, and the minutes until the zone reaches 4.5 people per m² at the current inflow. When a zone needs attention, an agent (gpt-oss-120b on Amazon Bedrock) checks that the camera has not moved, counts the crowd a second way, finds where in the zone people are pushing, measures the exit flow, and forecasts every action the venue allows. It puts one proposal in front of a steward: a sentence to read over the radio that says where the pushing is, a camera crop of that spot, the predicted peak with and without the action, and how far off that forecast was when tested. The steward approves or rejects. Ninety seconds later the system measures again, and escalates if the curve has not bent. After the evening, the safety officer reads a report generated from the hash-chained record, which states first whether the record is intact. Other agents can read the same measurements through a read-only MCP endpoint; approvals stay with the steward.

How a zone that needs attention reaches the steward

From measurement to decision. OpenCV 5 measures every zone; deterministic tiers open an episode; the agent investigates with its tools, and code checks every proposal before a steward approves it. Ninety seconds later the system measures again.

How I built it

  • OpenCV 5 vision: frames reduced once with resize (area) and undistorted with remap at half size, head detection (cvtColor, inRange, connected components), DIS optical flow, head-plane homography (USAC MAGSAC), Helbing's crowd fields with GaussianBlur on a 0.1 m grid, people counted across a gate line, pushing hotspots from connectedComponentsWithStats on the floor grid cut out of the image with warpPerspective, DM-Count on the OpenCV 5 DNN engine as a second counter, and a two-stage camera watchdog: findTransformECC (homography model, still-pixel mask) every 3 s, confirmed by ALIKED + LightGlue from the OpenCV 5 features module.
  • Physics: the continuity equation over each zone (dN/dt = Q_in − Q_out) gives the inflow and its trend from measured quantities; integrated against exit capacity measured on 24 real bottleneck runs, it gives minutes to danger and the effect of each action.
  • Agent: deterministic tiers decide when to investigate; the model picks tools, compares actions and writes the radio message; code refuses actions outside the venue's catalogue, proposals that cite someone else's forecast, radio messages with numbers no tool returned, and proposals that skip a gentler action that is as good. If the model is unreachable, a deterministic planner calls the same tools.
  • A record built to be audited: I audit controls for a living, so every decision can be reconstructed. Each trace entry is hash-chained and signed with an Ed25519 key the service can read but not write, and CloudWatch Logs keeps a copy the instance can only append to. The evening report checks both before anything else. Outside the public demonstration, an approval needs a named steward's PIN, with lockout after five wrong tries; the public demo runs in demonstration mode and its decisions are marked "no PIN".
  • Interfaces: the steward console, an evening report rendered from the trace, and a read-only Model Context Protocol endpoint serving the agent's own measurement tools.
  • AWS: one Graviton instance (t4g.small, Amazon Linux 2023), headless OpenCV 5.0.0 aarch64 wheel, the DNN engine chosen per architecture (classic on Graviton, 5.5 times faster there), JuPedSim compiled from source, Caddy with a Let's Encrypt certificate, no SSH (Systems Manager only), an instance role limited to Bedrock, one log group and one bucket, and a hash-chained trace mirrored to CloudWatch Logs.

Results (all reproducible from eval/)

  • The pushing index, from video alone and with no training, separates pushing from walking in trained raters' labels with AUROC 0.798 over 9,260 rated person-seconds (6 Jülich runs, 342 people), and 0.69 to 0.86 within each density band, where a head count carries no signal. The console's own code, at the half resolution it runs at, scores 0.797 on the same rows.
  • Together with density, the index reaches AUROC 0.831 against 0.786 for density alone (+0.045, 95% interval 0.031 to 0.060, leave one run out): it carries motion information a count does not.
  • Outflow counted from video: 56 of 56 people, error 0.04 people/s.
  • Minutes to danger: warning 3.6 to 4.1 minutes before the zone reaches 4/m², median timing error 32 s (5 simulated evenings).
  • On 7 more venue layouts it was never tuned on: 18 of 18 runs that reached 4/m² warned 3.2 to 5.2 minutes ahead (median error 35 s), and in 21 of 21 runs the proposed action was within 0.05 people/m² of the best.
  • With the console's own alert tiers, over 30 simulated evenings on 10 layouts: all 22 evenings that reached 4.5 people/m² were warned 4.1 to 7.6 minutes ahead.
  • Choosing actions on predicted peak plus measured error: mean regret 0.01 people/m² out of sample, against 1.85 on the forecast alone.
  • Camera watchdog: 84 of 84 rolls, zooms and tilts above 0.25 m caught, 0 false alarms in 132 checks; the ECC screen measures the floor shift to 4 mm in 228 ms on Graviton, and ALIKED + LightGlue (3,046 ms) run only when it sees motion. On the live instance a knock is flagged within 5 s.
  • Counting on oblique street CCTV (CroHD): DM-Count QNRF 10.9% error.
  • Agent: 20 of 20 test episodes reached the right kind of decision with gpt-oss-120b. In all 6 pushing episodes it located the pushing (1.4 m before the gate) and said where in the radio message.
  • Speed on the t4g.small: 528 ms per camera-second for the console's pipeline (718 ms at full size). DM-Count runs in 1.7 s on the classic DNN engine against 9.5 s on the new one; in the live console the agent's second count went from 75 s to under 6 s.
  • Accessibility: axe-core reports no WCAG 2.1 AA violations on the console (desktop and phone width) or the report.

Pushing index against trained raters, by density band

The pushing index separates pushing from walking within every density band, where a head count carries no signal. AUROC against trained raters' labels, 9,260 person-seconds from 6 Jülich entrance runs, no training.

Seven venue layouts the forecast was never tuned on

Seven street layouts the forecaster was never tuned on, three runs each. Left: every run that reached 4 people/m² was warned 3.2 to 5.2 minutes ahead. Right: regret of the chosen action, priced with the error measured on the first venue (blue) and on the forecast alone (grey).

Challenges I ran into

  • Seeking the replay per frame cost 75 ms and could abort FFmpeg; reading forwards cost 3 ms.
  • Measuring outflow by integrating field flux read 40% low; counting matched heads across a line matched all 56 people.
  • The forecast was accurate for actions on the whole inflow and about 2 people/m² optimistic for partial ones (one gate, one extra door). Rather than tune it, every proposal now carries the error each action's forecast made in testing.
  • ALIKED + LightGlue need about 1.6 GB while they run. On a 2 GB instance, each heavy check runs in a fresh child process, one at a time.
  • Watching the live console showed the agent's second count taking 75 s and its camera check 23 s: the two models competed for 2 vCPUs and 2 GB. A per-stage benchmark found that OpenCV 5's new DNN engine runs DM-Count 5.5 times slower than the classic engine on Graviton (it is the faster one on x86), so the engine is now picked per architecture; DM-Count sees only the zone's bounding box, and an ECC alignment screens the camera before the 3 s feature match.
  • The first camera screen used a rotation-plus-shift model and read a 2% zoom as steady. A homography model measures zoom and tilt as well.

Accomplishments I'm proud of

A physics index that agrees with trained human raters without any training, measured in metres and seconds, and an agent whose every number can be traced to a tool result in a tamper-evident record.

What I learned

Driving the real console found defects that unit tests missed: a stale forecast judged at approval time, a camera knock nobody noticed because a proposal was pending, actions forecast with mismatched data, and a 2 vCPU instance starved by two models at once. Measuring each stage on the target machine changed a design decision that a desktop benchmark would have hidden.

What's next

Calibrate on a real venue camera (the oblique calibration from floor markings is implemented and unit-tested), validate the pushing threshold on more experiment series, and add upstream cameras so the forecast sees surges before they reach the zone.

Data credits

Forschungszentrum Jülich pedestrian data archive (CC BY 4.0): 2018 entrance experiments, 2022 pushing ratings, 2021 bottleneck experiments. CroHD (CC BY-NC-SA 3.0) for evaluation only.

Built With

Share this project:

Updates

Submission history