Blurr

Homepage
Blurr works with photos too!
Multiple mask options

Inspiration

Anyone who posts street footage, bodycam clips, a vlog or a school event has to hide bystanders' faces and license plates first. Today that means one of two things: scrubbing through an editor and keyframing boxes by hand, or uploading the footage to a cloud service, which is the opposite of protecting the people in it. We wanted redaction you can ask for in plain language, check frame by frame, and run without a cloud.

What it does

Blurr hides people, faces and license plates in video (and photos) from a plain-language request.

Ask in plain words. "Blur all plates and the person in the red hoodie around halfway." "Everyone except the woman." "Pixelate faces in the first 10 seconds." "Unblur the plates." "Undo."
Or draw a box around anything, and it's tracked through the clip.
See what the agent did. Each reply lists its steps in plain words: the yes/no question it asked about each person ("Is this a man wearing a red hoodie?"), how many said yes, no or unsure, and what it masked. A coverage strip shows when each target is masked.
Preview before you export. Source shows every box the detectors found (solid if masked, dashed if not), and Program shows the frame as the export will render it.
Fix any mask by hand. Mask, unmask, restyle (blur, pixelate, black box) or drag a box to the right place and it's re-tracked from there. Every change is a version, and undo is one click.
Fail closed. If the vision model is unsure about a person, or the tracker loses someone, they stay blurred. Errors mean more blur, never less.
Photos too, exported with no metadata (no GPS position, camera serial or capture time).
Fully offline. Inference never touches the network.

How we built it

The agent never touches pixels. It edits a mask timeline (which object is masked, when, and with which effect) through a small set of typed tools, and a deterministic renderer applies that timeline to the video.

Scan at upload. A background scan finds faces and plates with Meta's EgoBlur models and people (and any other named object) with YOLOE, and tracks them across frames.
Understand the request. Plain requests are parsed by a template in milliseconds, with no model. Anything else goes once to Qwen3-VL-8B, which writes the whole plan as JSON. Plain code validates and runs it. The model never handles track ids.
Check who's who. "The woman" becomes a yes/no question that the same Qwen3-VL model answers about each tracked person, from crops across the track. Above 0.65 is yes, below 0.35 no, in between unsure, and unsure stays masked.
Render. Plain code applies the timeline: blur, pixelate or a solid box, with padding, interpolation between detections and a hold after a track is lost.

Built on one AMD GPU with ROCm. PyTorch 2.9.1+rocm7.13, transformers, Ultralytics, OpenCV, FastAPI and a single-file web UI. We built and measured on a Radeon AI PRO R9700 (32 GB), then moved to a Radeon 8060S Strix Halo (64 GB shared) when the Dublin hub went down. Same ROCm, same code. Detection, the 8B vision-language model and the planner all share one card: 17.9 GB peak on the R9700, 19.3 GB on the Strix Halo. fp16 cut each EgoBlur call about 3x with the same boxes (0.147 s to 0.049 s per 1080p frame on the R9700, 0.376 s to 0.129 s on the Strix Halo). On the R9700 a 720p clip scans at 1.12x real time, 24x faster than a 64-thread Threadripper PRO CPU. On the Strix Halo the full face + plate + person scan runs at 105 ms/frame at 720p.

Private by construction. ./run.sh puts Hugging Face and Ultralytics in offline mode, and an egress guard refuses and logs every outbound connection or DNS lookup. The log stays empty through our demo, and /health reports "offline": true.

Challenges we ran into

Checking our own benchmark. On the night-crowd clip most failed requests were label errors: face boxes half a face off, a plate box left on a frame after the plate drove away. We corrected the 28 boxes we could verify, left every ambiguous one as drawn, and report both scores (13/20 before, 14/20 after).
Plate false alarms. At a low threshold EgoBlur flags tail lights, signs and shoes. Requiring two confident detections took one clip from 232 plate tracks to 49 while keeping every real plate.
Small, fast plates. A 13 px plate moving 10 px between checked frames doesn't overlap itself, so the tracker split it. Linking by size and distance and starting masks a few frames early took plate coverage from 72% to 94.6% of frames.
Step-by-step agents are slow. Our first LLM loop picked one tool at a time and took over two minutes for "everyone except the guy in the red hoodie". One JSON plan per request, run by plain code, does it in 7 s.
Fragmented person tracks. The tracker sometimes loses a pedestrian and starts a "new" person. A stitcher joins tracks by position, heading, size and color (15 to 11 person tracks on one clip). We checked every join by eye, because merging two different people is worse than leaving a piece.
ROCm quirks. MIOpen couldn't read its kernel database on the cloud image, so every new crop size triggered a ~15 s kernel search per VLM call. Fixed-size crops plus MIOPEN_FIND_MODE=FAST fixed it. Asking the model for only the last position's logits avoided a 16 GB memory spike.
Moving GPUs mid-hackathon. The Dublin hub went down on Sunday. We re-measured on the Strix Halo and kept the numbers separate rather than copying them across.

Accomplishments that we're proud of

It really is offline. An empty egress log after a full demo, enforced by code, not by promise.
Fail-closed behavior we measured. With fail-closed on, 14/20 real-clip requests pass; with it off, 11/20. A keyword-only baseline gets 7/20.
Plates (Strix Halo): 100% of plate frames masked on a 1080p street clip and 93.7% on a 720p one, with 2.5% and 0.06% mean over-blur.
Labeled boxes at least 90% masked (Strix Halo): faces 0.95, plates 0.98, people 0.99. Our face labels are head boxes, so this undercounts faces.
An honest benchmark. Every number comes from one script over hand-labeled real clips, and we published our label corrections.
Four people, one contract. The typed mask timeline let us build the agent, detectors, renderer and UI in parallel against a stub, and the full test suite runs in about 15 s with no GPU.

What we learned

Keep the model out of the pixels. A typed mask timeline between the agent and the renderer made every result checkable, undoable and testable without a GPU. Measure before you optimize: the GPU, not decoding, was the limit, and fp16 was the win. And treat benchmark labels as suspect before blaming the model.

What's next for Blurr

Live mode: a standing rule ("blur any plate that appears") on a stream behind a short delay buffer, then an OBS virtual camera. On the Strix Halo, detecting every 5th frame at 720p behind under 0.8 s of delay would keep every mask in place, but it isn't built yet.
Pixel-accurate masks with SAM 3 instead of padded boxes.
Fewer vision-model checks per request in crowds.
More categories: screens, on-screen text, documents.
Better recall on dim, motion-blurred faces in masks, which EgoBlur misses even at a 0.05 threshold.

Limits: no nudity or graphic-content detection, masks are padded rectangles rather than outlines, offline speed on the Strix Halo is 0.4x real time, and photos support JPEG, PNG, WebP and BMP but not HEIC.