Consent

Inspiration

Wearable cameras make recording effortless, but people in the background rarely get a say. We asked: what if a bystander could request privacy directly, without needing the recorder’s approval?

Consent explores that idea: your visibility, your choice.

What it does

Consent lets a person control whether their face appears clearly in our live video output.

Cover both eyes deliberately to activate face blur.
Make a peace sign to reveal your face again.
Use spoken requests such as “Blur my face” or “I give consent.”
Leave and return while keeping your session privacy preference.

Protection changes the actual image pixels, not just a dashboard overlay. The system produces a live protected preview and simultaneously records original and protected videos. Original footage is available as a clearly labeled developer diagnostic.

Our main voice demonstration supports one visible person. Multilingual requests use the same privacy state, with English, Mandarin, Russian, and Spanish intent handling. Speech recognition reliability varies, particularly for Mandarin.

How we built it

Our camera path uses a live iPhone camera through Meta’s iOS Device Access Toolkit and Mock Device Kit. A Swift bridge receives frames through the SDK’s camera-stream callback and sends them to our Python backend.

The backend combines OpenCV, MediaPipe, pretrained face detection and recognition models, and a session privacy registry. Gesture detection, identity matching, face protection, and simultaneous recording share one integrated pipeline.

For voice, the Mac microphone feeds Meta Muse Voice Transcribe. Muse Spark interprets the recognized request into a validated structured action. That action updates the same person registry used by gestures.

A React and TypeScript dashboard displays the protected stream, privacy states, actual transcripts, and recordings.

We used the actual Meta SDK with simulated glasses and a live iPhone camera. Physical Meta glasses were unavailable; this prototype does not enforce protection across native Meta recordings.

Challenges we ran into

The hardest problems appeared between components: associating a hand with the correct person, recognizing someone after they returned, and keeping privacy preferences separate from whether someone was currently visible.

Variable frame arrival also exposed gesture-timing failures. We adjusted temporal checks to require multiple valid observations without assuming a constant frame rate.

Voice added another challenge: a partial transcript or misunderstood negation must never become permission to reveal a face. We kept strict action validation and treated transcription failures separately from intent interpretation.

Accomplishments that we're proud of

We connected live iPhone frames through Meta’s SDK to our existing privacy engine and web preview.

We demonstrated gesture and voice controls updating the same person’s privacy state, remembered preferences after leaving and returning, and paired recordings with protection applied during capture.

We also built a persistent application-side Meta API budget and kept API credentials entirely on the backend.

What we learned

Privacy is a continuing preference, not a one-frame detection. Reliable behavior requires separating identity, visibility, and a person’s current request.

We also learned to verify each layer independently. Correct interpretation of written Chinese does not establish reliable Mandarin speech recognition, and passing an automated test does not replace a live demonstration.

What's next for Consent

Next steps are testing with physical Meta glasses, broader evaluation across people and environments, and improving multilingual speech reliability.

We also want to evaluate crowded scenes and speaker-to-face association. The current voice demo deliberately assumes one visible speaker; it does not claim multi-person speaker identification.