Inspiration

We saw the dangers of online deepfaking, especially those that were explicit and non-consensual, and the effects that it had on the mental well-being of online citizens
We also recognized that locating and reporting deepfakes can be inaccessible to many
According to the New York Times, Grok has created and shared 1.8 million sexualized images of women The law surrounding how to take this type of content down requires the exact identification of the exact deepfake, and we thought it was overly complicated for such a severe issue.

What it does

The user will upload their information and selfies of themselves onto the website (where they are securely used and deleted afterwards)
The platform automatically scans social media feeds and identifies images which exhibit facial similarities
Once images have been sourced, the platform runs our machine learning model on those faces to detect deepfakes.
The platform then returns to the user a sequence of (initially censored) deepfakes of themselves, which have already been flagged as likely to be deepfaked
The user can then flag these posts to the social media platform with an in-built dashboard that guides the user on how to report the issue
If the images have not yet been removed by 48 hours, the platform will prompt the user via text to escalate the issue with the FTC

Users enroll with 3-5 selfies, which are converted into one face embedding and then deleted. Each selfie must match the others to prevent impersonation.
The platform scans X for posts with the X API connected to the user or deepfake-related terms, then face-matches any images and videos it finds.
Matching content is analyzed using a detector trained on frozen DINOv3 features from 23,000 real and synthetic images.
Each match also receives reconstruction, Content Credentials and NSFW checks to help assess authenticity and legal relevance.
Reposts are grouped using perceptual hashes, so the user only needs to confirm each image once.
A secure agent tracks confirmed cases, drafts notices, checks whether posts remain live and texts the user when action is needed.
The dashboard prepares the information required to report the content to X and escalate the case to the FTC.

CLIP and DINOv3 performed well in validation but struggled with 49 edits from the previously unseen Grok Imagine generator.
LoRA fine-tuning achieved the highest validation score but overfit to our phone photos, so we shipped the simpler and more reliable DINOv3 probe.
To limit false positives, the detector uses a high-confidence threshold. This reduces recall, so its score is treated as supporting evidence and the user makes the final decision.
Scans run in the background to avoid Modal’s request timeout, with only confirmed face matches sent to the heavier models.
An hourly job and dashboard check restore any 48-hour case timers interrupted when Modal shuts down an idle container.
Because X and the FTC accept reports only through web forms, the dashboard prepares every field while leaving the legally significant final submission to the user.

The whole path works: selfie, scan, confirmation and notice, then a Droid session that waits out the 48 hours, rechecks X and builds an FTC packet if the post is still up, with a text to the user at each step
We tested on a person and a generator the models had never seen, and shipped the model that held up there over the one with the best validation score
The Act’s requirements are enforced in code, and no notice gets generated without the signed authorization, both of the user’s confirmations, a named signer and a contact address
The agent’s guardrails live in its tools: it can only touch its own case, gets one X check and two texts per session, and can’t escalate unless check_live saw the post still up after the deadline
Selfies are deleted after enrollment, media from X is never stored, and evidence stays blurred until the user chooses to look

The model with the best validation AUC was the worst one on a teammate it had never seen, so a held-out person now decides what we ship
Detectors do much worse on real posts than their benchmarks suggest (one study we read found open detectors lose about 45% AUC on real-world 2024 images), so the user’s confirmation has to be the final decision
Guardrails hold up better in tool code than in a prompt, because the agent can still pick a wrong step and the tool will refuse it
On a serverless platform anything with a deadline has to live in the database and get woken up by a scheduled job, since the server can go to sleep at any time
The model ended up being the smaller part of the work, and most of our time went into making a legal process usable for someone on one of the worst days of their life

Improve recall on unseen generators, starting with the 23 Grok edits our current high-precision threshold missed, while continuing to test every release on entirely held-out people.
Expand monitoring beyond X, beginning with Reddit and Bluesky, and group reposts of the same image across platforms using PDQ hashes.
Add frame-level video analysis so long videos can be face-matched and evaluated without running the full detector on every frame.
Move case state from a single SQLite-backed Modal container to a durable database and job queue that can support concurrent scans and deadlines at scale.
Work with X and the FTC toward direct reporting integrations, while keeping the user responsible for reviewing and submitting every legal statement.
Build a consented real-world benchmark covering more phones, compression artifacts and generators so deployment performance—not validation AUC—determines what we ship.