Safe Frame

The master passed. The version people watched was never checked. Safe Frame sweeps a whole catalogue in ClickHouse to find the flash risk a conversion introduced, and the profile that caused it.

Comment

Inspiration

A film is approved once. Then it becomes dozens of versions: frame-rate conversions, tone-maps, downscales, social crops, ad-break inserts, burnt-in subtitles.

Photosensitivity testing happens at ingest, before any of that. That is standard practice and it is correct, because a defective master multiplies across the whole delivery ladder. But it runs before the transforms, and a per-file check run after a transform has no memory of the parent, so it cannot tell you whether a violation is new.

The version an audience actually watches can carry a risk the approved master never had. Nothing in the normal pipeline asks that question.

The stakes are documented. Photosensitive epilepsy affects roughly 1 in 4,000 people, about twice as often in women, and usually begins before age 20 (Fisher et al., Visually sensitive seizures: an updated review by the Epilepsy Foundation, Epilepsia 63(4), 2022). In 1997 a single broadcast sent 685 children to hospital in Japan. The obligation is real too: WCAG 2.3.1 and Ofcom Section 2.

Excellent tools already exist for judging a file: EA IRIS, Apple's VideoFlashingReduction, the ffmpeg photosensitivity filter, Harding-style broadcast analysers. We searched and could not find a documented check that compares a rendition against its own approved parent and attributes a newly introduced violation to the transform that caused it. That is a gap in public documentation, not proof that no private integration exists. So we built it.

What it does

Safe Frame is a master-to-rendition photosensitivity regression pre-check. It asks one narrow question: did this conversion introduce a flash risk that the approved master did not have?

  • Aligns on presentation time, never frame index. A 24 to 60 fps conversion renumbers every frame but preserves pts, so frame index is not a valid lineage key.
  • Evaluates the published criteria inside ClickHouse in a single pass, then isolates child-only violations in the same query.
  • Attributes every regression to the transform that produced it.
  • Draws both tracks on one shared scale, master above and rendition below, so "the master stayed under the criterion for its whole runtime and the rendition did not" is read off measurements rather than asserted.
  • Turns findings into a QC brief with a multi-step Gemini agent that cannot decide anything.
  • Checks your own video in the browser. It is decoded locally, never uploaded and never played back at you, and only downscaled samples are sent.
  • Public API with no credential required. Every endpoint works anonymously.

On the live corpus of 424 titles, 3,392 renditions and 10,176,000 transition measurements, the sweep returns 66 renditions that introduced a violation, 43 general flash and 23 red flash, in about two seconds.

The rule most tools miss

Published guidance is not one rule. We implement two of the three tests, every threshold quoted from WCAG 2.3.1.

Test Threshold in Safe Frame
General flash luminance delta 0.10 or more, and only when the darker image is below 0.80
Red flash red delta 0.20 or more, with no luminance condition at all
Affected area 25% or more of the screen
Rate more than 6 opposing transitions in 1000 ms
Spatial pattern not implemented, and named on the site rather than left implied

The missing luminance condition on the red rule is the most consequential decision in the detector. WCAG gives saturated red its own test because people are more sensitive to red flashing. A saturated red alternating with a colour of matched relative luminance moves luminance almost not at all, so gating the red rule on luminance recreates the exact blind spot the separate test exists to close.

On our corpus, 23 of the 66 findings are red-flash only. A luminance-only workflow misses every one of them.

The two rules are also windowed independently, so four red-qualifying and four luminance-qualifying transitions in one second is eight transitions and still not a violation of either rule.

From a count to something a team can act on

Sixty-six findings is not sixty-six problems.

Encoder profile Renditions Regressed Rule
60fps_interp 424 22 general flash
adbreak_insert 424 21 general flash
subtitle_burnin 424 12 red flash
social_crop_v 424 11 red flash
three other profiles 1,272 0 none

Four profiles out of seven account for everything, and the two rules cluster on different profiles: the frame-rate and ad-break paths introduce luminance flashes, the crop and subtitle paths introduce saturated-red ones. Two root causes, two owners, four upstream configurations to fix instead of sixty-six outputs to patch, and fixing them stops the next unsafe version being made.

How we built it

ClickHouse, through the official MCP server. Every catalogue read and every live verdict goes through ClickHouse/mcp-clickhouse in read-only stdio mode, against a self-hosted ClickHouse 26.3 LTS cluster on a dedicated GCP VM with a SELECT-only user. The criteria are evaluated in SQL, not read from a precomputed column: the 1000 ms window is a RANGE BETWEEN CURRENT ROW AND 999 FOLLOWING, and direction is encoded so that min(dir) != max(dir) is exactly "both directions present". All three tools the server advertises are called, because list_databases and list_tables prove the session reaches a real cluster with a real schema, which run_query alone cannot.

ClickHouse Agent Skills. We worked through all 31 official rules against our own schema. Most were applied. Four were declined with measurements showing why, including one where the recommended single-scan shape measured 27 percent slower on our data, because per-granule min/max already skips about 93 percent of the second scan.

Google Cloud. Two Google ADK agents on Gemini 2.5 Flash via Vertex AI. The triage agent has four tools, every one a live ClickHouse query through the same MCP server, and it sequences them itself: survey the sweep, profile every transform to find the systemic cause, size the red-flash blind spot, then go deep on the pair it ranks first. The tool-call sequence is returned with the brief, so the multi-step work is checkable rather than trusted. Cloud Run, Secret Manager and Cloud Logging carry the rest.

Gemini has no path to a verdict. It reads database evidence and writes prose. Pass or fail is decided by SQL, every time. If MCP fails or the count cannot be parsed, the API returns 502 rather than guessing.

Challenges we ran into

Proving it is not circular. A synthetic corpus invites one fair objection: bursts were planted, then found. We answered it by recovering the planted set independently, from the generator's own hashing decisions, using a query that reads no measurement column at all. Two queries produced by independent means; agreement between them is a result rather than a restatement.

Result: 66 planted, 66 found, precision 1.000, recall 1.000, and all 86 decoys correctly rejected.

The decoys are what make the number worth anything, because recall alone can be bought by flagging everything. Seventy carry a real flash burst and are still not regressions, because their master has the same burst at the same presentation time. Seventeen clear every floor but sit above the published darker-image ceiling. One rendition is in both sets, which is why seventy plus seventeen is eighty-six.

Trusting a claim we had never actually verified. Our SQL and our Python detector are two implementations of one safety rule, and we asserted they agreed. The proof needed a live cluster, CI had none, so it silently skipped everywhere anyone could look. When we finally made CI stand up a real ClickHouse, the comparison turned out to be broken and could never have passed. The implementations did agree once it was fixed, but for weeks nothing demonstrated it. A check that cannot run is a claim, not evidence.

An optimisation we had to refuse. ClickHouse's own guidance says a fixed answer that many people reload should not be re-derived per request. We built the refreshable materialized view and it works: 44 rows in 5.9 ms instead of 10,263,552 rows in 932 ms, identical output, about 158 times faster. We deliberately did not enable it, because our site claims nothing on it is precomputed, and serving that button from a five-minute-old view would make the claim false. The DDL ships so a real deployment can create it in one statement.

Not becoming the thing we are warning about. This is a site about flashing content, so nothing on it animates, no clip ever autoplays, and the sample files that do contain a deliberate flash sequence sit behind a warning and an explicit press.

Accomplishments that we're proud of

  • A question that, as far as we can document, nobody was asking, answered at catalogue scale in about two seconds.
  • Precision and recall of 1.000 against a ground truth the detector cannot see, scored separately on the 400 authored titles and the 24 whose every value was measured from real pixel arrays. Both cohorts agree, which is what rules out the corpus doing the work.
  • The red-flash blind spot, demonstrated rather than described: a clip a luminance-only checker passes and ours fails.
  • Findings reduced to four fixable upstream configurations.
  • Every threshold traced to a published definition. Our 25% area condition follows the broadcast and ISO family rather than an assumption of our own, per Jordan and Vanderheiden, International Guidelines for Photosensitive Epilepsy: Gap Analysis and Recommendations, ACM Transactions on Accessible Computing 17(3), 2024. And the one rule we did not implement is named on the page rather than quietly omitted.

What we learned

The most useful thing we learned is about evidence rather than video.

Every impressive number in a hackathon project is a claim until someone can check it, and the checks are the first thing to rot. Ours skipped in silence for weeks while a green badge sat on top of them. So we ended up building the product so that a stranger can verify it without trusting us: press a button and the query runs in front of you, with the elapsed time reported; the stack panel earns each status by making a real round trip rather than reading a config file; the API answers with no credential so nothing is gated behind our goodwill.

We also learned that the right use of a partner tool sometimes means declining one of its features and saying why, with the measurement attached.

What's next for Safe Frame

  • Real footage. Everything here is measured against a self-authored synthetic corpus. The obvious next step is a pilot against a studio's actual delivery ladder, which is the only way to learn the real regression rate per profile.
  • The spatial pattern rule, the third published test we have not implemented.
  • Decode in the pipeline. Today the service measures frames you supply. Wiring it directly to a transcode farm would make it a gate rather than a check.
  • Alert on the profile, not the file. If one encoder profile starts regressing, that is the thing worth paging someone about, before the next hundred renditions ship.

Built With

Share this project:

Updates

Submission history