Signal Survives

• 

Inspiration

Modern generated images no longer need obvious visual mistakes, while social platforms routinely recompress, resize, crop, blur, add noise and recolour images. A detector that succeeds only on pristine outputs from familiar models creates false confidence.

We therefore treated Track 5 as a robustness and generalization problem: rank authentic and fully AI-generated still images while actively trying to expose source, format, geometry and content shortcuts.

What it does

Signal Survives accepts a directory of JPG, PNG, WebP, BMP or TIFF files and writes one ordered JSON row per image containing the image path and a continuous AI-positive score. It uses one checksum-pinned PE-Core-L vision model with 315,776,001 parameters, safely below the two-billion-parameter limit.

We deliberately do not claim a universal yes/no threshold. The official metric is ROC AUC, the hidden class balance is unknown, and a detector score should support human review rather than claim authorship, deception or fraud.

How we built it

We froze the public Apache-2.0 PE-Core-L encoder and trained only a 1,025-parameter binary head. Our balanced 13,574-row training mixture spans authentic scenes, people, objects and low-resolution images, alongside GAN, older diffusion, Stable Diffusion, SD3, FLUX and other recent generated-image sources. PixArt and Imagen were excluded from training and used only in frozen evaluation holdouts.

Every training view receives either no degradation or one workshop-listed transformation. We also used exact and canonical deduplication, generator-held-out tests, authentic-source rotation, and prompt/content pairing.

The shortcut problem we found

Our raw data was almost perfectly separable without looking at pixels because formats and dimensions leaked the labels. That would have produced an impressive but misleading result.

We applied the same EXIF correction, square crop, resize and JPEG-q96 encoding to both classes. The pixel-free metadata AUC fell from 0.9984 to 0.5131, while literal PNG and square-shape rules fell to 0.5. This is the evidence that changed our approach: we stopped asking only “how high is the score?” and started asking “what signal produced it?”

The organizer's demo-only 4,998 COCO val2017 and 8,843 DALL-E Advanced images were used zero times for training, tuning, model selection, calibration or thresholding. Recorded non-commercial training rows are also zero.

Evaluation strategy

We applied all 19 workshop transformation settings individually and computed the workshop score: 50% clean ROC AUC plus 50% pooled transformed ROC AUC. The final model choice followed a rule committed before either candidate saw the final 1,024-image source-coherent gate.

Frozen audit	Clean AUC	Transformed AUC	50/50 score	Weakest condition
CIFAKE matched source	0.9228	0.8920	0.9074	0.7945
Modern semantic pairs	0.9982	0.9941	0.9962	0.9848
Open Images source rotation	0.9819	0.9700	0.9759	0.9397
Community Forensics, 78 model names	0.9502	0.9473	0.9487	0.8234
Final NTIRE source-coherent audit	0.9907	0.9837	0.9872	0.9308

On the final gate, the selected PE model beat the fixed PE/DINO blend by 0.0510 on the 50/50 score and by 0.0824 on the weakest condition. These are frozen development results, not a prediction of the unpublished organizer score.

Challenges we ran into

The hardest problem was validation illusion. A model can appear excellent by learning PNG versus JPEG, square versus non-square, a dataset's subject matter, or the collection pipeline rather than generation. A DINO branch helped one narrow low-resolution slice, then failed badly after we rotated the authentic source. That attractive ensemble was rejected.

Heavy Gaussian noise, human-made art, realistic generated photographs and unseen collection pipelines remain explicit limitations.

Accomplishments we are proud of

• We built an end-to-end, deterministic image-directory-to-continuous-JSON runner.
• We kept the selected system to one reproducible model rather than a fragile ensemble.
• We added causal codec and geometry controls instead of trusting aggregate accuracy.
• We preserved dataset provenance, licences, hashes and frozen promotion gates.
• We verified the exact source export and checkpoint locally on Apple MPS and on an NVIDIA Tesla T4.

What we learned

Clean accuracy is the easiest way to fool yourself. Robustness is not a final augmentation toggle; it is a measurement discipline. The most useful experiment was often the one that invalidated our favourite result.

At an illustrative, uncalibrated 0.5 cutoff, the final clean gate has 161/512 false positives but only 4/512 false negatives despite 0.9907 AUC. The strongest authentic false positive is a surreal human-made animal collage, while the weakest generated examples resemble ordinary travel and event photographs. That is why we expose a ranking score and make uncertainty visible.

Impact and practical use

Signal Survives can help moderators, fact-checkers and users prioritize images for review before they infer authenticity from appearance alone. It should never be the sole evidence of deception, authorship or fraud.

What's next

The next step is broader source-controlled evaluation against future generator families and real redistribution pipelines, followed by threshold calibration for a specific operating environment. The design remains intentionally auditable: when a new family or failure mode appears, it can be added as a frozen gate instead of hidden inside one headline score.

Built With

• apple
• computer-vision
• machine-learning
• nvidia
• pillow
• python
• pytorch
• scikit-learn
• timm
• torchvision