Inspiration
Crashes are rare, lagging signals. A dangerous junction announces itself for years first through conflicts: the hard brake, the gap that closed too fast, the driver who crossed a metre behind another. None of it appears in crash statistics, and no city can afford an engineer watching thousands of hours of video to find them.
What it does
It finds the near misses in ordinary overhead intersection video and hands a human traffic engineer a ranked review queue, with the clip, the trajectories and the numbers behind each one.
OpenCV 5 does all the perception, and every safety number is computed in metres on the road surface, never in pixels:
findHomographywith RANSAC for ground-plane calibration, with the reprojection error reported rather than assumed- MOG2 background modelling with resolution-scaled morphology and
connectedComponentsWithStats, filtered by physical area in m² - a constant-velocity
cv2.KalmanFilterper track with association gated in metres. The tracker is built from primitives because OpenCV 5 removedcv2.legacy,TrackerCSRTandTrackerKCF - Time-to-Collision solved analytically and sampled at fixed screening distances, plus Post-Encroachment Time from true path intersection
The agentic layer
A Strands agent supplies the triage, and the vision output genuinely changes what it does next. A cheap coarse sweep produces candidates and quality flags; a roughly 20x more expensive full-resolution pass is spent only where that sweep found something; each candidate is then verified by re-running with perturbed detector parameters, because a conflict visible at only one setting is a tracking artefact; and only what survives reaches a human.
Five BeforeToolCallEvent hooks enforce the rules in code rather than in a prompt: no fine pass before a coarse pass, no escalation without verification, a compute budget, a severity floor, and a step cap. The agent cannot escalate an unverified conflict even if it decides to, because the hook cancels the call. Every decision is traced in evaluation/agent_session.json, and the workflow diagram is in the repository.
How we evaluated it
Against exact ground truth. Synthetic sequences establish measurement accuracy: precision and recall 1.00 on 4 true conflicts and 2 deliberate true negatives, TTC mean absolute error 0.040 s, tracking RMSE 1.02 m. Real CC BY 4.0 drone footage over Songdo establishes that it survives real imagery, at 3.3 fps on full 4K. One command regenerates evaluation/results.json, including the machine and OpenCV version, so every published number is checkable.
The evaluation found four real defects that intuition missed: phantom conflicts between fragments of one vehicle, PET misapplied to following traffic, a scale-blind area filter, and 4K vehicle over-segmentation. Each is documented in the report with its cause and its fix.
Limitations, stated rather than hidden
Minimum separation is overstated and is reported two ways rather than relied on: 5.49 m mean error from observations alone, 2.70 m when Kalman-coasted samples are included, because converging vehicles merge into one blob at exactly their closest approach. PET is unmeasurable on the tightest encounters for the same reason. The threshold sweep cannot discriminate on only six scenarios. Severity is a triage ranking, not a crash probability.
AWS: designed, never deployed
The AWS design was never applied. No AWS account was obtained: the compute-grant round closed before this entry was registered, and attaching a personal payment card to a cloud account for a student competition was not acceptable to us.
What exists is infrastructure-as-code that was never run: S3, a container-image Lambda, API Gateway with a 2,000 requests/day quota, reserved concurrency 5, CloudWatch alarms and an AWS Budgets deny action at $20, in infra/template.yaml, with the procedure in docs/deploy.md. There is no stack, no URL and no password, and the README and report section 9 say so. The architecture is Arm-ready and the per-stage 4K baseline is measured, but COOL was never run on Graviton, so that award is not claimed.
The rules offer a working endpoint or an arranged live screen-share, and we are taking the screen-share. Everything runs locally in about five minutes.
Challenges we ran into
OpenCV 5 removed the legacy tracker API, so the tracker had to be rebuilt from Kalman primitives with association gated in metres rather than pixels. That turned out to be the better design anyway, since a pixel gate means different things at different points in the frame.
The harder problem was trusting our own output. The first version was confidently reporting conflicts between two halves of one bus. Measuring against exact synthetic ground truth, rather than eyeballing overlays, is what surfaced that and three other defects.
What we learned
A measurement you cannot check is a guess with a decimal point. Building the synthetic suite first, with known answers, cost a day and found four bugs that looked fine on video.
What's next
Calibration from map imagery so a city does not need to mark ground points, a larger annotated conflict set so the thresholds can be fitted rather than set, and the deployment, if an account ever exists.
Data and privacy
Real footage: Songdo Traffic, Zenodo 10.5281/zenodo.13828383, CC BY 4.0, Fonod, Cho, Yeo and Geroliminis (EPFL / KAIST). The licence was verified via the Zenodo API before use, and candidate datasets carrying NonCommercial or NoDerivatives terms were rejected. Details in docs/DATA.md.
Footage is shot from 140 to 150 m, where individuals are not identifiable and plates are not legible. The pipeline stores trajectories, not imagery; pedestrian-scale regions are blurred before any clip is written; there is no face or plate recognition and no re-identification. Output is a review queue for a human engineer: no enforcement, and no automated decision about any individual.
AI assistance disclosure
Claude Code was used as a coding assistant. Per the rules, using an AI assistant to write the entry does not by itself constitute the agentic workflow; the agentic component claimed here is the runtime perception, decision and action loop described above and traced in evaluation/agent_session.json.
Log in or sign up for Devpost to join the conversation.