RedactedBytes

Cover
Feature 1
Feature 2
Boarding Ticket Original
Boarding Ticket Blurred
Man and License Plate Original
Man and License Plate Processing
Man and License Plate Blurred
Bank Statement Original
Bank Statement Blurred

Inspiration

Have you ever snapped a photo or recorded a video, only to realize later that you captured more than you intended? A stranger's face in the background, a license plate on the street, a confidential document on a desk, or a QR code on a wall. These seemingly harmless details are Personally Identifiable Information (PII). In the age of AI, these data points are dangerous. A name tag can lead to a LinkedIn profile, a QR code can expose a private link, and a license plate can be traced to a home address. Malicious actors can use AI to quickly correlate these shards of information to trace, profile, or impersonate individuals. We aim to flip the script. Instead of hoping your data stays private, our tool automatically finds and censors PII before it's ever shared.

What it does

RedactedBytes is a powerful, privacy-aware media pipeline that empowers creators and platforms to automatically protect privacy at scale. It leverages cutting-edge multimodal AI to: Detects Sensitive Elements: Identifies a wide range of PII in images and videos, including faces, ID cards, passports, license plates, credit cards, signage text, and QR codes. Applies Smart Redaction: Automatically applies pixel-perfect blur, pixelation, or blackout effects to sensitive areas. Processes Any Media: Supports both batch processing for images and pre-recorded videos, and real-time streaming for live video feeds. Protects Your Data: Offers flexible deployment options, including local processing, to ensure sensitive data never leaves your device.

How we built it

Our system is built for performance and flexibility: AI Core: Fine-tuned a state-of-the-art YOLO-based segmentation model (yoloe-11l-seg) with a custom taxonomy of over 100 PII and PII-adjacent object categories. We use open-vocabulary prompts to easily generalize to new classes like "passport" or "signboard text" without retraining. Cross-Platform Deployment: Converted our model to ONNX and CoreML formats, enabling high-speed inference on Python backends, cloud servers, and iOS/Edge devices. Backend: A robust FastAPI service handles file uploads, orchestrates inference, and returns redacted media via base64 encoding or secure signed URLs to handle large files. Frontend: A modern, user-friendly interface built with Next.js, Tailwind CSS, and shadcn/ui for seamless upload, preview, and download. Video Engine: A custom OpenCV + ONNXRuntime pipeline processes video frame-by-frame in real-time, applying masks and effects efficiently to maintain high FPS.

Challenges we ran into

Ambiguous Classes: Defining "sensitive text" required a nuanced approach. We expanded our class list to cover multilingual scripts (Latin, Chinese, Arabic, etc.). Performance Trade-offs: Balancing redaction quality (retina_masks) with processing speed was key. We found an optimal setting for real-time performance.

Accomplishments that we're proud of

Built a cross-platform PII redaction pipeline from the ground up in under a week. Achieved real-time performance (~25 FPS) for 720p video on a consumer GPU. Developed a flexible, comprehensive taxonomy of sensitive objects tailored for real-world privacy concerns. Created a seamless user experience with a powerful model underneath, demonstrated through a polished full-stack application.

What we learned

Streaming is crucial for long-form video and live feeds to avoid memory bottlenecks. Class granularity is critical: Too broad leads to over-censoring; too narrow leads to dangerous leaks. True innovation lies in optimizing the entire end-to-end pipeline, not just improving model accuracy metrics. Deploying advanced models on mobile (CoreML) requires careful planning and an understanding of hardware constraints.

What's next for RedactedBytes

OCR Integration: Add a secondary layer of protection to find and redact sensitive text (numbers, names) even if the primary object detector misses it. Live Stream Protection: Extend our low-latency engine to work on RTMP feeds for platforms like Twitch or Zoom. TikTok/Instagram Plugin: Build a plugin for social media creators to preview and apply redactions before uploading content. On-Device Mobile Redaction: Investigate and deploy pruned, lightweight models (e.g., YOLO-Nano) for full privacy-focused processing directly on smartphones. Differential Privacy Metrics: Develop metrics to quantitatively measure and guarantee the quality of our redaction for auditing purposes.