Robust Detection of AI-Generated Images Under Real-World Transformations

Track 5 submission — frozen-backbone AIGC detection engineered for corruption robustness, unseen-generator transfer, and faithful explainability.

Overview

We built a detector that scores a folder of images for the likelihood each is AI-generated (predict.py → JSON of {image_path, pred}), designed around the three failure modes that break detectors in production: post-processing (JPEG, blur, resize, noise, color, crop), generators unseen at training time, and dataset shortcuts that inflate lab numbers. The final model is a frozen DINOv2-B/14 backbone read at two depths (blocks 8 and 10) by a pair of ~10M-parameter transformer-encoder heads whose logits are averaged — far under the track's 2B cap, reproducible from scratch in ~2 hours on one GPU.

Results (held-out generator; corruption grid per track spec): clean AUC 0.9978 · robust mean over the six families 0.9943 · track score (0.5/0.5) 0.9960 — on the complete demonstration set (4,998 reals + 8,843 DALL·E fakes). Transfer to generators never seen in training: GigaGAN 0.9772, Imagen 0.9847 — the two-depth ensemble beats either member on both axes. Full per-condition tables: docs/ROBUSTNESS_SUMMARY.md.

1. Dataset Construction and De-Confounding

1.1 Corpus. 78k images: fakes from 30+ generators; reals deliberately multi-source (SID_Set, LAION, COCO, ImageNet, FFHQ, LSUN-church). Materialized as versioned ZIP shards with byte-exact ledgers, pHash-deduplicated, with leak-checked splits asserted in every training driver. Evaluation uses a held-out generator plus unseen-generator OOS sets; the track's WildFake demonstration subset was never used in training.

1.2 Shortcut audit. A classifier reading image resolution alone scores ~1.0 on the raw public data — the labels leak through resolution, sharpness, format, and source. We quantified and closed two confounds: source×label (real-share gap 98.8pp → 22pp; retraining the unchanged architecture on the de-confounded corpus produced the largest single improvement of the project: +0.011 unseen-generator AUC, +0.008 robust mean) and content×label (k-means over frozen-CLS embeddings showed content bins ranging 31–94% real; inverse-imbalance reweighting was evaluated and bought nothing over the de-confounded corpus, so the unweighted sampler ships).

2. Preprocessing and Augmentation Protocol

Information-level normalization (fair_eval_normalize): every image is pushed to a common ~200px information level (downscale + JPEG + upscale) at train and eval, reducing the resolution oracle from ~1.0 AUC to chance.
Class-symmetric corruption training: one corruption per image per step, sampled from a class-independent distribution with label-blind seeding; severity ranges extend beyond the evaluation grid; operators are drawn from a multi-implementation bank with an implementation-level train/test split, so the model is robust to the transform family rather than one library's artifact signature.
Per-batch random resolution (392–644px) prevents fixed-grid anchoring.

3. Model Architecture and Loss

3.1 Intermediate-depth readout. Per-patch evidence mapping revealed that DINOv2's final layer homogenizes patch evidence (every patch echoes the image verdict). Reading block 8 of 12 — and truncating the backbone there — improves holdout AUC (+0.010) while cutting ~33% of backbone FLOPs. A depth sweep confirms the shape (block 4: 0.93; ~block 8: peak; final: trailing), and capacity scaling inverts: ViT-L underperforms ViT-B by 0.03 at matched protocol — detection evidence is shallow, and deeper abstraction discards it.

3.2 Head and loss. A 2-layer transformer encoder with fixed 2D sin/cos positional embeddings (zero-initialized residual) emits an image logit and signed per-patch logits. Training: BCE with label smoothing ε=0.05 (+0.002 robust mean) and an optional feature-consistency term — cosine alignment of pooled representations across each clean/corrupted pair — which makes the evidence corruption-stable (+0.0017 robust). The contrastive variant (InfoNCE) was tested and rejected: its uniformity term rewards content-specific features and cost 0.016 transfer AUC.

3.3 Complementary-depth ensemble. The shipped model averages the logits of two independently trained heads reading blocks 8 and 10 — decorrelated views of the same frozen backbone (per-image correlation 0.86). The ensemble strictly dominates either member: +0.002 clean and +0.003 robust over the better single head, with the deeper member carrying more of the hardest (configural) fakes.

3.4 Falsified alternatives. Each rejected knob remains in src/config.py with the measured verdict that killed it: six pooling-head architectures; masked-reconstruction residual scoring (I-JEPA-style, chance-level); MIL top-k image logits (three variants — the head's self-attention broadcasts the verdict into every patch, so top-k pooling collapses to global pooling); resolution/token-granularity scaling in both directions; warp-based anatomical augmentation (teaches the warp artifact, not anatomy).

4. Explainability

The per-patch evidence maps functioned as an engineering instrument with verifiable impact: they diagnosed the final-layer homogenization (changing the readout layer), exposed the verdict-broadcast mechanism (closing the MIL branch), and quantified per-corruption evidence collapse (motivating the consistency loss). The maps ship in predict.py --explain as a capability preview — corruption-stable by training, but region-level verification against annotated defects is stated future work.

5. Error Analysis and Limitations

Configural fakes remain the hardest class (GigaGAN 0.977 ensemble vs ~0.99 for diffusion-family generators); heavy Gaussian noise (σ=0.10) is the weakest corruption cell (0.955; mechanism analyzed in docs/ROBUSTNESS_SUMMARY.md). Details, FP/FN populations, and trade-offs: docs/ERROR_ANALYSIS.md.

Development Tools

VS Code + Claude Code · vast.ai GPU instances · Weights & Biases experiment tracking

Libraries and Frameworks

PyTorch · DINOv2 (frozen backbone, Apache-2.0) · torchvision · pandas / pyarrow · scikit-learn · NumPy · Pillow

Datasets and Assets

SID_Set and WildFake (both track-suggested; WildFake supplies the LAION / COCO / ImageNet / FFHQ / LSUN-church real partitions and most generator fakes) · Community Forensics PublicEval fakes (Park & Owens, CC-BY-4.0, used with attribution) · self-generated corruption test sets per the track parameter grid. Model stack (code, weights, DINOv2 backbone) is MIT/Apache-licensed; per-source training-data licenses are documented in the repo.