image-detection

A 316M-parameter detector (16% of the track's 2B cap) that holds up through JPEG re-encoding, blur, rescaling, noise, colour jitter and cropping: AUROC 0.9971 clean, 0.9912 averaged over the 15 transform cells, 0.9986 on 400 held-out generators, 397 of them families it never trained on.

The insight: the obvious evaluation lies

On the validation split, a classifier that reads only JPEG quality, never a pixel, scores AUROC 0.9998. Headline numbers on that split measure file provenance, not detection. So we treated robustness as an evaluation problem first: every reported number sits beside a zero-pixel null model, and a bias-corrected protocol re-encodes both classes to a common size and JPEG quality before each scored transform, so the resize and noise cells stop rewarding metadata tells. Our first system scored 0.478, below chance, and was killed by its own bias probe; the shipped one was chosen purely by pre-registered measurement on held-out generator families.

The system

Two linear heads, 62 KB together, on one frozen PE-Core-L/14-336 tower (PyTorch + timm). One head trains on older generator families (WildFake fakes vs LAION, FFHQ, ImageNet and COCO reals), the other on modern commercial generators (FLUX, Midjourney, Firefly, Ideogram, from Community Forensics and SID-Set). Both pools are JPEG-normalised so no head can learn compression tells, and hash-firewalled against the validation split. Serving fuses the two calibrated heads: 0.44 s/image on CPU; retraining a head takes minutes.

Results

Trade-offs, stated plainly

This is a ranking signal for a platform integrity queue (human review, down-weighted distribution), not an auto-takedown. The shipped threshold is precision-heavy because falsely accusing an authentic creator is the expensive error: 0 false positives on 5,000 COCO reals, catching 41% of DALL·E 3 fakes. The ranking is far stronger (TPR 0.958 at 1% FPR), and a published threshold ladder gives the measured trade at five other cuts. Known limits: noise is the weakest transform, frontier commercial generators are the hardest case, and only photographic reals were measured.

Solo entry, built end-to-end as an autonomous Claude Code research loop. Every table regenerates from persisted per-image scores in the public repo.

Built With

• claude