Inspiration
I used to watch a lot of IRL streams on Twitch, and I've seen streamers get swatted. Sometimes all it took was a street sign behind them, or the number on a door. Streamers can't watch every frame while they're walking and talking to chat, so I built something that does.
What it does
IRL Shield is a relay that sits between your phone and Twitch.
- Your phone streams with Moblin, or any app that streams RTMP.
- The relay holds the video for about a second.
- In that second it finds readable text, like street signs, shop names, house numbers and plates, and paints over it.
- Only then does the frame go out to Twitch. If a frame hasn't been checked in time, it goes out protected.
A stream delay alone doesn't hide anything, because someone still has to spot the sign and react. A fixed blur covers one spot on the screen. IRL Shield finds the text as it moves, inside that one second.
Detectors miss things, so people stay in control.
- A mod or the broadcaster types
!shield holdin chat, and the whole stream switches to a privacy screen. Only the broadcaster can bring it back with!shield resume. A hold can't take back frames already sent, so it's for what you can see coming. - Chat votes on the mask look with
!skin barand!skin patch. - The streamer has a private phone dashboard that shows the shield's state and has its own Hold button.
How we built it
Codex wrote the code. One Codex builder thread (GPT-6.1 Sol) and the helper agents it started made more than 110 commits across 26 merged pull requests. Claude Opus planned the milestones, reviewed the changes and tried to break them.
One example: the stream kept arriving with torn frames before masking even ran. Codex traced it to the media server's write queue overflowing on large keyframes. It showed the tear at queue size 128 and a byte-exact clean stream at 1024, then shipped the fix in commit 940223a.
The pipeline runs in this order.
- Moblin sends RTMP over Tailscale into MediaMTX.
- PP-OCRv5-mobile text detectors run on CUDA, a fast pass at 8 Hz and an accurate pass at 4 Hz, on a delayed frame clock.
- The relay paints the masks, NVENC encodes the frame, and a TLS-verified connection sends it to Twitch. The stream key stays in memory.
Chat commands arrive through Twitch EventSub.
How we measured it
The detector never grades itself. Tesseract and Apple Vision decide what text was readable in the input. Those engines plus PP-OCR then check whether any of it was still readable in the output.
We tuned on one set of clips. A separate set of fresh Creative Commons footage stayed locked until the settings were frozen, and we ran it once. It was six 5-second clips from three sources.
With the earlier PP-OCRv6 detector, 106 of 126 natural text intervals stayed hidden (84%), offline. The median masked area was 13.6% of the frame and the 95th percentile was 50.4%. The synthetic control scored 14 of 22, because one clip failed our check that the rest of the picture stays real video. We aimed for 90% and didn't reach it, and we report that rather than tuning on the test set.
The shipped PP-OCRv5-mobile detector hides 127 of 146 natural text intervals (87%) and 49 of 54 synthetic labels on our tuning clips. It also cuts the worst-case masked area from 77% to 15% of the frame. These are tuning-set results from a two-judge screen, not a held-out test. We switched on the last day because it put far fewer stray masks on windows and trees in real street footage.
Challenges we ran into
- Small, distant text causes most misses. On our tuning clips, 12 of the 13 leaks we measured were 20 pixels tall or less.
- Some settings that caught more text also covered huge parts of the frame, so a guard on masked area rejects them.
- We tried letting a local language model un-blur harmless text like "OPEN". Two blind annotators labeled 5,902 text readings from street footage and agreed only 4 were safe to show. On real streets nearly all readable text could point to a place, so the stricter default stays.
Accomplishments we're proud of
- It runs live on Twitch on real street footage, with chat and mod control.
- Its numbers come from tests it couldn't see beforehand, judged by OCR engines other than its own detector.
What we learned
For a privacy tool, the honest default is to hide more. The most useful feature turned out to be the hold, which puts a person in charge when the detector misses.
Limitations
- It covers readable text only. Landmarks and other visual clues stay visible.
- It needs a Linux PC, ideally with an NVIDIA GPU. A CPU mode exists.
- Live publishing is configured for our test channel. Local testing and bandwidth tests work without it.
What's next
- An optional second pass, where a vision model checks each covered region and decides whether it actually points to a place. The shield could then hide less of the screen and catch what the detector misses. It would be opt-in, since a hosted model costs money and can't run on every frame.
- A detector trained on street signs, storefronts and house numbers at small sizes, where most misses happen.
- Smaller, subtler masks, and settings for what gets hidden.
- A way for a mod to flag a miss so it becomes a mask instead of a full-screen hold.
- An OBS plugin, plus Mac and Windows support.
Credits
- Montreal (Plateau) street walk by Andre Carrotflower, CC BY-SA 4.0, Wikimedia Commons.
- Music is "Wetlands" by HoliznaCC0, CC0 1.0, from the Free Music Archive.
- The demo video is released under CC BY-SA 4.0 because it adapts CC BY-SA footage.
Log in or sign up for Devpost to join the conversation.