Stradiva

Rear-end collision on a roadside camera: speeds and the road grid recovered from the lane paint alone, with no calibration
Could braking have prevented it? A verdict only if it holds for every reaction time and braking level in the outlined box
Near-miss measures and reconstructed speeds up to the moment of contact, each with a 90 % interval
Camera knocked mid-clip: Stradiva finds the bump with ECC alignment, re-analyses each part, and the refusal stands
Added mode: one minute of real dashcam footage from comma2k19, measured from the moving car
The vehicle ahead against the car's own radar, which Stradiva never saw: 2.4 % median distance error
Wrong assumption caught: the lane geometry contradicts freeway markings, so Stradiva refuses speeds that would be 2x too high
Architecture: OpenCV 5 does every image operation; on AWS an S3 upload triggers a container Lambda and DynamoDB tracks the job
Held-out results, failures included: each test set fixed in advance and run once, every number from committed result files

Inspiration

Collisions are filmed more than ever: traffic cameras, shop CCTV, dashcams. That footage almost never becomes measurements. The camera is uncalibrated, nobody surveyed the scene, and the questions people actually need answered are physical ones. How fast was each car going? When did the lead car start braking? Could the following driver have stopped in time? Today those answers come from a reconstruction expert days later, or from someone eyeballing the video.

I wanted a system that answers those questions from the footage alone. Just as important, it should say plainly when the footage can't answer them. A confident wrong speed in a collision case is worse than no number at all.

What it does

Upload collision footage from an ordinary, uncalibrated camera. Stradiva works out the physics and states every assumption it needed along the way:

Speed of every tracked vehicle, the lead vehicle's brake onset, and the moment of contact
Near-miss measures: smallest gap, time to collision, and braking needed, each with a 90 % interval
"Could braking have prevented it?" Stradiva doesn't pick one reaction time. It replays the approach for every reaction time from 0.75 to 2.0 s and every braking level from 5 to 8 m/s². It says "avoidable" or "not avoidable" only if the answer holds across that whole box. Otherwise it names the assumption that decides it.

Which assumption decides the answer

It cross-examines itself. After every reconstruction, a separate set of checks tries to contradict it with evidence the fit never used: markings predicted from each other, the geometry re-estimated on each half of the clip, dash proportions against the assumed standard, vehicle widths, and physically possible motion. When a check blocks a result, an investigator runs the remedy for that specific failure. For a camera that was knocked mid-clip, for example, it finds the bump with OpenCV's ECC image alignment, splits the clip and analyses each part again. The remedy's result has to pass the whole cross-examination again from scratch. If it doesn't, the refusal stands, and Stradiva tells the user which input would help (frame rate, focal length, marking standard). It never relaxes a threshold to produce an answer.

Added mode: dashcam footage. For a moving camera, Stradiva measures the car's own speed from the markings sliding past. It also measures the distance and closing speed of the vehicle ahead, using a YOLOX-S detector on OpenCV 5's DNN module and two independent range rulers: where the tyres meet the road, and how wide the car looks. If the two rulers disagree, that window is refused.

Stradiva never assigns fault. It measures what the footage supports, and says why when it can't.

How we built it

The road paint is the ruler. A static camera looking along a road sees painted markings, and three ideas from projective geometry turn them into a metric road plane with no calibration:

The edge lines and centre line meet at the vanishing point. Because the three lines are equally spaced, the centre line is the harmonic conjugate of the edges, which gives the horizon and camera roll in closed form.
Along any 3D line, inverse image distance to the vanishing point is affine in road distance: $$\frac{1}{d_{\text{img}}(x)} = a\,x + b$$ So sampling a strip uniformly in $ 1/d $ with cv2.remap turns perspective-compressed dashes into an exactly periodic pattern. Fitting the dash centres fixes the along-road scale, given the dash cycle length.
Vanishing point, horizon, lateral spacing and the along-road map determine the image-to-road homography. Bootstrapping the dash fit gives a distribution of homographies, not just one, so the uncertainty propagates into every speed.

OpenCV 5 does every image operation: HoughLinesP and fitLine for the markings, remap for the inverse-distance strips, createBackgroundSubtractorMOG2 + KalmanFilter + Hungarian assignment for tracking, calcOpticalFlowPyrLK and findTransformECC for camera motion, phaseCorrelate for dashcam strips, and dnn.readNet with YOLOX-S from the OpenCV Model Zoo for the vehicle ahead. On top of that sits a road-plane kinematics layer (RTS smoother with propagated pixel noise), event detection, a Monte Carlo avoidance envelope, and the cross-examination and investigator loop.

Delivered on AWS, defined in one SAM template:

The browser gets a presigned URL from API Gateway + Lambda and uploads straight to a private S3 bucket.
The upload triggers a container-image Lambda (arm64, OpenCV 5 inside) that runs the full analysis.
DynamoDB tracks the job.
Footage expires automatically after 7 days (results after 30).
CORS is locked to the site's origin.
The Next.js front end is a static export on Vercel. A live upload takes about 45 s end to end.

Evaluation was built like an experiment, not a demo:

The splits were fixed in advance.
Code was developed on the dev split only, then frozen and tagged.
Each held-out set was run once.
Real-footage tests were pre-registered in writing before they ran.
The scoring punishes confident errors, not abstention.

Challenges we ran into

The one number a single view can't give you. The dash cycle length sets the speed scale, and it can't be identified from one static view: longer dashes seen from farther away look identical. So I made it a stated prior with ±3 % uncertainty, attached it to every result, and tested what is observable, the dash proportions.
A pre-registered test that failed. On a real city-street dashcam holdout, speeds came out about 2× too high because the freeway marking standard was wrong for that street. I published the failure. Afterwards I added a second, independent ruler (lane width plus focal length), and Stradiva now refuses that case instead of reporting wrong speeds.
Range. Beyond about 70 m, one image row covers 0.7 m of road or more, so contact can't be established within the vehicle-length uncertainty. Stradiva now says so instead of guessing.
No legal real crash footage with ground truth. I couldn't find any I could legally use, so I built a simulator with exact ground truth. The collision verdicts are validated on simulation only, and the product, the video and the docs all say so.
Cloud details. Lambda's memory cap, and presigned uploads that failed in the browser until I switched the S3 client to SigV4 signing.

Accomplishments that we're proud of

Real dashcam footage against the car's own radar (pre-registered, run once): 2.4 % median distance error, with the radar inside the 90 % interval in 17 of 17 answered windows. Closing speed was within 0.58 m/s (median). Stradiva never saw the radar.
Road geometry: on the same 18 scenes, the first version accepted 4 with up to 9.3 % scale error. The final version accepts all 18 with at most 0.23 %.
Reasoning layer with true tracks: 0 harmful errors on the newest held-out split.
Failures published:
- From raw pixels on that split it got 5 of 30 right and made 5 harmful errors, mostly where the following car was hidden.
- The investigator changed no collision outcome on held-out data. Its value today is the trace and the input requests.
Refusal as a first-class result. "Not reconstructable", "contact not established" and "disputed" each come with the reason and the input that would help.
Live end to end on AWS, with a public web UI where every number on screen comes from committed result files.

Real dashcam: the vehicle ahead against the car's own radar

What we learned

Saying "I can't tell" is a feature. The most useful output was often the refusal plus the reason, not another number.
Test in layers. Feeding the reasoning layer ground-truth tracks separated its errors from tracking errors. That showed tracking at long range, not the physics, is the weakest link.
Pre-register, or you'll fool yourself. Writing down the test before running it is what turned the city-street failure into a fix instead of a quietly dropped result.
A second, independent ruler beats a better single ruler. Two measurements that must agree catch wrong assumptions that no single measurement can.

What's next for Stradiva

Fix the failure family found on held-out data: contacts extrapolated while the following car is hidden.
Stronger tracking at range in fixed-camera mode, using the same DNN detector the dashcam mode already uses.
Real collision footage with ground truth, through partnerships with road-safety agencies or reconstruction firms, so the verdicts can be validated beyond simulation.
More collision types: angle and side impacts, and pedestrians.
Learning the marking standard per jurisdiction, so fewer users need to supply it.

Links: Live demo · Code · Technical report · Every number