Rolling Shutter Clock

Inspiration

Video provenance is usually attacked with cryptography: sign the file at capture, verify the signature later. That only works if someone signed it, which for almost every clip that matters — a leaked recording, a insurance claim, a piece of citizen journalism — nobody did.

But there is a signature in the file that nobody chose to put there.

Mains lighting flickers at twice the grid frequency, 120 Hz in North America. The grid is not a metronome: supply and demand push it around constantly, 60.01 Hz then 59.98 then 60.00, and that wander is effectively identical across an entire interconnect and never repeats. A camera with a rolling shutter exposes the sensor one row at a time, which means it samples that flicker row by row. The banding you see in a video of a screen or a lit room is a recording of the electrical grid, sampled by the image itself.

Strip the metadata and it is still there, because it is in the pixels.

What it does

Takes a metadata-free, mains-lit clip and answers two questions: when was this shot, and was its timeline cut.

1. Extracts per-row mean luminance across every frame into a one-dimensional signal at row rate.
2. Removes scene content — the hard part, since moving objects and camera motion also change row luminance.
3. Estimates instantaneous frequency near 120 Hz over sliding windows using a quadratically-interpolated FFT peak with phase unwrapping across windows for sub-bin resolution, producing an ENF trace with uncertainty.
4. Cross-correlates that trace against a full 24-hour grid reference and reports the best-matching time, the runner-up, and a confidence from the peak-to-sidelobe ratio.
5. Runs the same match in sliding windows to find splices — a discontinuity in matched time is a cut.

It also handles the aliasing case explicitly: when the row rate undersamples 120 Hz the observed peak is an alias, and the app computes and displays that mapping rather than silently reporting the wrong frequency.

How we built it

Vanilla JavaScript and canvas, no DSP library. The clip generator implements a genuine rolling-shutter model — row r of frame f is exposed at t = f/fps + r × (readout/height) and its brightness is modulated by the mains illumination integral — so the banding in the test clips is physically derived rather than drawn on. Controls expose true capture time, fps (including the nasty 29.97-vs-30 near-aliasing case), readout ratio, modulation depth and noise, and a spliced mode that concatenates two different times.

Challenges we ran into

The estimator worked long before the system did, and the reason took a while to see.

Early evaluation was dismal: about 30% of clips within 60 seconds. But the clips that did lock were accurate to a fraction of a second, which is a strange failure mode — an estimator that is either perfect or hours wrong is not a noisy estimator, it is a matching problem.

It was clip length. A 45-second ENF trace is simply not unique across 24 hours of reference; there are thousands of places it fits almost as well, so the correlation peak never separates from its sidelobes and the confidence gate correctly refuses. Sweeping duration made it unambiguous:

• 45 s → 0 of 6 clips within 60 s
• 90 s → 4 of 6
• 150 s → 8 of 8, with zero error

Nothing about the signal processing changed. We raised the default to 150 s and widened the UI control to 30-240 s so the dependence is explorable rather than hidden, because it is the single most important operational fact about this technique.

Accomplishments we're proud of

Measured over 24 generated clips spanning modulation depth and noise:

• median absolute timestamp error 0.18 s
• clean subset: 100% locked, 100% within 5 s, median error 0.075 s
• 66.7% within 60 s across all conditions, including deliberately degraded ones
• splice located within 2 s of the true cut
• when the peak does not separate, the verdict is INCONCLUSIVE and no time is printed

That last one matters more than the accuracy figure. A forensic instrument that occasionally refuses is usable in an investigation; one that always answers is not.

Honest limits

The 24-hour grid reference is synthetic. It is a realistic random-walk model standing in for a real FNET/GridEye feed, and the loader is structured so a real CSV drops in — but no claim here has been validated against a genuine grid recording, and that is the single biggest gap between this and a usable tool.

The clips are generated, not filmed. The rolling-shutter model is physically derived, but real footage adds compression artifacts, auto-exposure, auto-white-balance, and scene motion far messier than what we simulate. The scene-content removal is the component most likely to degrade on real video.

33% of evaluated clips are not identified, concentrated at low modulation depth and high noise — dim scenes, LED lighting with its own driver frequency, and heavily compressed footage. Those come back INCONCLUSIVE, which is correct behaviour and still a limit.

Finally, this recovers when, not where: ENF is shared across an entire interconnect, so it can distinguish Eastern from Western but not one city from another.

What we learned

The signal processing was the fun part and the correlation was the real problem. We spent days improving an estimator that was already accurate to a tenth of a second, when what the system needed was more of the signal it already had. Being able to look at a bimodal error distribution and read "this is matching, not estimation" is the most transferable thing we took away.

What's next

Ingest a real FNET reference feed, test against genuinely filmed footage under several lighting types, add LED-driver frequency detection, and quantify how much compression the trace survives.

Built With

• canvas
• digital-forensics
• dsp
• enf
• fft
• javascript
• media-verification
• power-grid
• rolling-shutter
• signal-processing