Cleansheet

Per-viewer sponsor inventory for Fire TV: computer vision finds every gambling and alcohol board in a match and replaces or blurs it, for each viewer.

Comment

Inspiration

Live football is wall-to-wall betting sponsorship, much of it on the perimeter boards. For someone in gambling recovery, betting ads are a relapse trigger. Many Muslim viewers avoid gambling and alcohol for their faith. Parents watch with their kids. None of them can choose what the stadium shows. Broadcasters already swap boards per region with virtual advertising; we wanted to do it per viewer, on the TV.

What it does

Cleansheet is a sports app for Fire TV (Vega OS). While a match plays, a Sponsors button in the player opens a D-pad navigable panel: for Gambling and for Alcohol, the viewer chooses Show, Replace, Blur or Auto.

  • Replace warps a family-friendly sponsor onto every hidden board, in perspective, locked to the board as the camera pans and zooms, within one video frame of the picture.
  • Blur covers the board with a blurred image of itself.
  • Auto replaces the boards the detector is confident about and blurs the rest, so a shaky detection never shows
  • Settings are saved per profile in the cloud, so each person in the household gets their own stream. gets their own stream.
  • Which replacement sponsor fills each board slot comes from an API, not the app: that's the inventory a leagued can show a betting brand toone viewer and a family brand to another, so every board slot can be sold twice.

How we built it

  • Computer vision, offline. A YOLO pose model (yolo11n-pose, trained with Ultralytics on an Apple Silicrd's box, brand and fourcorners. A tracker adds what one frame can't: camera-motion prediction, rebuilding corners cut off by the screenbrand and confidence per board so it never flickers between treatments.
  • **Training data without hanictional sponsors onto thereal perimeter boards of public-domain broadcast footage (Venezuelan league matches on Wikimedia Commons), so eveground truth. A segmentationmodel puts players back in front of the pasted boards, and each clip is re-compowith different brand orders shis spot is the bettingbrand". - **A sidecar track, like capver runs the model. Thepipeline publishes a compact JSON track of every board's corners for every frameto the HLS stream. The video e stream serves every viewer.
  • The Vega app. React Native 0.83 on Vega OS, forked from Amazon's vega-sports-app sample. TheeoSurfaceViewthrough the W3C Media API; overlay views sit on top. The app readscurrentTime, looks up the boards for the frame on screeent with a 4x4 perspectivematrix` transform. We measured sync with frame numbers burned into a test streawith a 0.117 s offset the oved frame or one frame off.
  • AWS (one CloudFormation stack, us-east-2). S3 (private) behind CloudFront (origin access control, Brotls the HLS streams, sidecartracks, blurred board textures and art; compression takes a 12 s clip's sidecar from 1.07 MB to 231 KB. A Lambda fis the sponsor-inventory API:per-profile settings and the sponsor for each board slot. A $5 budget alert guarit, and it costs nothing at d Accuracy (held-out clip fdium and lighting, 360frames): recall 99.9%, precision 97.9%, brand accuracy 99.4%, median corner erropx at 1280x720. ## Challenges we ran into - Sync. Vega exposes `cur-frame timestamp, and on theVirtual Device it trails the picture by 3–4 frames in whole-frame steps. We builburned-in frame-counter streaated a fixed offset.
  • Blurring video from an overlay. React Native's blurRadius only blurs an image, not what's behind a vis a blurred, straightenedtexture for every board, and the app warps it like a replacement.
  • **Boards cut off by the fra across the screen until thetracker rebuilt hidden corners from neighbouring boards. Later, near-degenerate quads drew as screen-wide triangleshortening check on both thepipeline and the app.
  • **No brands we could legall invented, and the footage ispublic domain.

Accomplishments that we're proud of

A perspective-correct, frame-synced overlay on a TV platform using only standard React Native and W3C Media APlises to a stadium it neversaw; and a product framing (per-viewer inventory) that gives leagues a reason to ship it rather than fight it.

What we learned

How far declarative transformctive warp per board at 30fps), how to measure A/V sync without a frame callback, and that synthetic compositing gives exact labeld brand placement to stop amodel memorising positions.

What's next for Cleansheet

  • Real broadcasts: a one-class board detector trained on real footage, then identify each tracked board on image classifier) and map it to a category with a confidence that drives Auto.
  • Live: run the same worke sidecar a few seconds behind the stream.
  • Put players back in front oith a per-frame person mask.
  • Sign-in for private per-viewer settings, and a self-serve portal for leagues to sell board slots.

Built With

Share this project:

Updates

Submission history