Inspiration
Problem
Whenever my boss sees me scrolling reels, he gets really mad.
Solution
The obvious fix is to stop scrolling reels during work have my computer have my back. When boss is approaching, mute and get the reels off screen for me.
Hands-free boss evasion :)
A maybe less silly version of this problem is shoulder surfing: folks watching your screen when you're in a Cafe, library, or open office. Opp detector is built for both cases: targeting specific folks and keeping an eye out for everyone.
Since it's watching your webcam all day, it's really important that it keeps what it sees to itself: so it's fully on-device (and this meant solving a lot of fun efficiency problems :) )
What it does
Opp Detector is a small macOS app. You add people, from a photo or by snapping them straight from the live camera, and tick the ones who are opps. Then you start the detector and get back to whatever you shouldn't be doing.
- A live, mirrored camera preview boxes everyone it sees: red for opps, green for people it knows, grey for strangers. A big banner says CLEAR, or OPP IN FRAME with their name.
- The moment an opp walks in, it mutes your Mac and switches to another desktop. If it can't switch desktops, it hides the front app instead.
- Once they've been gone for a few seconds, it unmutes and switches back, so you're right where you left off.
- Privacy mode (one checkbox): a stranger standing close behind you, with at least two people in frame, counts as a threat too, so a shoulder-surfer gets the same treatment as your boss.
- You can add, remove, and opp/un-opp people while it's running, without restarting anything.
Who's it for? Consider an open-plan worker whose screen faces the walkway.
This was me over the summer, and the pain is real!
How I built it
I built in python (packaged w/ uv) in two processes:
- The detector runs in a child process. It reads the webcam with OpenCV, scales frames to 480p, and runs InsightFace's
PersonAnalysison them twice a second. That finds people and faces, matches faces against the registered photos, and falls back to body re-identification when no face is visible. It also sends a 480p JPEG preview to the UI about 10 times a second. - The app is a small Tk window. It talks to the detector over two one-way pipes: commands go down (stop, register, unregister) and previews/detections come back up.
- Trigger: someone counts as in frame at ≥ 0.5 similarity and ≥ 1% of the frame. The app keeps the opp count for the last 5 analysed frames and only fires on a rising edge, when the count jumps above where it was a moment ago, debounced to once every 5 seconds. An opp standing in the doorway doesn't keep re-triggering it.
- Evasion always mutes first (
osascript), then picks the best method the Mac's permissions allow:- With Accessibility, press ctrl+right arrow (or left on the last desktop), then check that the desktop really changed.
- Without it, or if that shortcut was turned off, hide the frontmost app.
- With no PyObjC at all, sleep the display.
- People live in a local
data/folder: a JSON file plus one JPEG each.
I tried a web frontend, then swapped it for a little Tk window. It was simpler and it's all I needed :)
Challenges I ran into
Among a TON of smaller items...
- The camera lies about its frame rate. Mine reports 15 but delivers 30, so I had to base the throttling in analysis off the clock, not frame count.
- MacOS silently drops synthetic keypresses if Accessibility is disabled. This was a painnnn to debug around, but ended up being only one extra check (
CGPreflightPostEventAccess()) to fix. Fun fact: the permission has to go to the terminal that started Python, not Python itself - I wanted to pause media, not just mute it. But MediaRemote is locked down since macOS 15.4, and without Accessibility the only other options are play/pause toggles that could just as easily start your music. So it mutes.
Accomplishments that I'm proud of
- The evasion degrades gracefully. It works out what this Mac will let it do, picks the best option, and checks its own work instead of assuming the keypress landed.
- Adding someone from the camera takes one click plus a name. They're marked as an opp and live immediately: the detector registers them on the fly instead of restarting, because a restart means reloading the model.
- It's all local. No account, no server, no frames leaving the machine.
What I learned
- How macOS permissions really behave: they attach to the parent app, they're fixed when the process starts, and denied actions often fail silently.
- A LOT about multiprocessing and Tk. I feel like I still have a while to go for mastering either, but figuring them out was really fun :).
- Don't trust hardware metadata. Measure measure measure.
What's next for Opp Detector
- Turn it into a proper menu-bar app, so Camera and Accessibility permissions can go straight to the app (instead of to Terminal)
- Allow multiple photos per person. The detector already accepts a list, but the app only stores one.
- Escalate by how long they stay / how close they are. I started building this out, but it ended up being too complicated for this MVP. It'd be a really cool feature to have though!
- Broadcast to the rest of the room: it'd be really cool to see a scaled up version of this where we use multiple co-workers cameras to triangulate the position of the boss in the room, then switch modes based on their calculated sightlines.
Credits
- The whole detector loop, multi-process plumbing, rising edge trigger, evasion strategies/fallbacks/etc. was me (some help from Claude for debugging, some frontend, and some of the evasion strategy impls).
- Facial recognition models: InsightFace 2.1 (MIT)
- Inference for said models: ONNX Runtime
- Webcam access, pre-/post-processing: OpenCV
- Image logistics (for frontend = load, save, preview, etc.): Pillow, PyObjC (for
Quartz,AppKit,Foundation,objc) - Packaging + running: uv
