AI Image Detection

Multi-branch training, consistency regularization, and meta-fusion pipeline
Confusion matrices across 15 transformation settings show how the detector performs on 13,841 images.
Our AI-image detector maintains strong accuracy across transformation, demonstrating robust performance under realistic conditions.
Complete multi-branch inference and linear SVM fusion pipeline.
These examples demonstrate how reducing images to 25% of their original size and upscaling them again destroys fine visual evidence.
Before-and-after examples reveal how strong Gaussian blur can erase authentic image details.
Cross-validation across multiple models and decision thresholds identifies the most reliable fusion strategy.

Inspiration

Existing vision models and AIGC detectors already understand real-world images. Tweaking their weights often ruins that knowledge and forces them to rely on fragile visual artifacts.

Base vision models are not broken. Instead of updating pre-trained weights, we froze the vision backbones completely and focused on making sense of the embeddings they produce by training a clean decision layer across multiple crops.

What it does

Point it at an image and it returns the probability that it is AI-generated, along with a Real or Fake verdict.

Three frozen vision backbones analyze the image, and a linear fusion model makes the final call:

Community Forensics (22M ViT): Evaluates low-level digital synthesis traces in a single pass.
DINOv3-B Stacking (86M): Extracts features across 3 crops, incorporating the forensic score.
DINOv3-L Detector (300M): Extracts deep semantic features across 8 crops.
Linear SVM Fusion: Fuses all branch outputs and flags any image with \(P(\text{fake}) \ge 0.70\) as fake.

The system runs locally via CLI, inside a notebook, or as a Kaggle GPU kernel. Inference spans 423M total parameters across the three frozen branches and classifier heads, completing well within standard cloud GPU budgets.

How we built it

A single model represents a single point of failure, and variants of a single architecture tend to share blind spots. Combining an untouched off-the-shelf forensic detector with two multi-crop frozen backbones decorrelates error distributions, making late fusion highly effective.

The results across 13,843 held-out validation images:

Branch	Solo Accuracy
Community Forensics	0.8149
DINOv3-L detector	0.9582
DINOv3-B stacking	0.9619
Fused (SVM)	0.9724

Paired consistency training. Every training sample passes through the model head twice: once pristine and once through a stochastic degradation chain. A combined focal loss and KL-divergence consistency loss penalizes divergence between the two views:

The clean representations are detached from the computational graph during backpropagation.
Gradients flow strictly toward the degraded view, preventing the model from minimizing consistency loss by degrading its clean representations.

Curriculum augmentation. Ten transform families scale in intensity from severity level 0 to 5 across training epochs. Mild perturbations dominate early training, while moderate and severe corruptions ramp up over time. A 10% floor on pristine samples prevents the classifier heads from drifting away from clean baseline distributions.

Robustness validation. Checkpoints are selected using mean AUC evaluated across a fixed grid of 14 discrete degradation benchmark conditions, rather than raw clean accuracy.

Test-time augmentation and fusion. DINOv3-L aggregates predictions across 8 crops, while DINOv3-B averages over 3 crops. A linear-kernel Support Vector Classifier (\(C=0.1\)) trained with probability scaling fuses the predictions.

Datasets and Assets Used

Training and Evaluation Datasets:
hy2628982280/WildFake: A collection spanning GAN, diffusion and other generator families. We ignored the coco and DALLE splits as they would be used for evaluation for the Hackathon.
saberzl/SID_Set: A collection of Real and AIGC-generated images from social media.
hy2628982280/WildFake's COCO val2017: Non-AIGC data for evaluation.
hy2628982280/WildFake's DALLE Advanced: AIGC data for evaluation.
Model Weights and Artifacts:
- DINOv3-L Detector Weights (best_robust.pt): Multi-crop classifier head checkpoint selected on the 14-cell degradation grid.
- DINOv3-B Stacking Head (best_robust.pt): Lightweight classifier head integrating frozen DINO representations and forensic scores.
- Linear SVM Fusion Model (selected_final_fusion.joblib): Serialized linear decision pipeline with the 0.70 operational decision threshold.
Integrity and Split Validation:
- SHA-256 and 64-bit DCT pHash Blocklists: Precomputed hash tables used during data ingestion to prevent exact and near-duplicate leakage between training and validation splits.

Challenges we ran into

The resolution shortcut confound. Evaluating raw dataset splits initially produced a 0.995 AUC. Metadata inspection revealed real images were downscaled to \(200 \times 200\) while AI-generated samples were \(1024 \times 1024\). The classifier was learning dimension artifacts rather than generative synthesis traces. Replacing downscaled reals with native-resolution COCO images dropped the score to a genuine 0.972.

Near-duplicate leakage. Re-encoded samples bypassed exact SHA-256 checks. Enforcing 64-bit DCT perceptual hash checks caught near-duplicates across splits.

Multi-crop determinism. Test-time crop sampling originally tied to batch index positions produced slight score variances across batch sizes. Keying crop generation to fixed per-image seeds resolved this variance.

Storage limits on cloud environments. Extracting features on the fly from streamed archive shards prevented out-of-memory and disk exhaustion errors on ephemeral instances.

GPU compute quotas. Freezing the vision encoders turned multi-hundred-million parameter training into a single forward feature-extraction pass, allowing SVM fusion optimization to run on a CPU in under two minutes.

What we learned

Inspect raw data distributions first: Resolution discrepancies can easily masquerade as state-of-the-art model performance.
Feature extraction dominates compute: Forward passes through frozen backbones account for the computational overhead; linear classifier convergence is negligible by comparison.
Calibrate thresholds to operational costs: Setting the operational boundary to \(\tau = 0.70\) yields 0.9958 specificity and 0.9592 recall, strictly minimizing costly false-positive accusations on authentic media.
Directional consistency matters: Robustness requires anchoring degraded representations to detached clean baselines during paired training.

Built with

Development tools

IDE: Visual Studio Code.
Programming Language: Python 3.11 using scripts .py for CLI usage and Jupyter notebooks .ipynb for visual demonstration.
Hardware: 2x NVIDIA Tesla T4 (Kaggle Free Weekly Quota) using kaggle CLI.
Hosting: GitHub.

Libraries and frameworks

Core ViT models training and inference: torch>=2.13, torchvision>=0.28, timm>=1.0.29, transformers>=5.4.0.
Core fusion model training and testing: sciki-learn>=1.9, xgboost (candidate fusion method), joblib>=1.5.
Source connection: huggingface_hub, modelscope, kaggle.
Data streaming and manipulation: requests, datasets, pandas, numpy.
Image processing: pillow,scipy.
Visualization: matplotlib, seaborn, tqdm.

License

DINOv3 License

What's next

Resolution-matched evaluation across emergent generator architectures (GigaGAN, GALIP, MAGE, Midjourney v6+).
Temperature scaling and Platt calibration for strictly reliable absolute probability estimates.
Out-of-distribution robustness benchmarking against unseen post-processing pipelines.