DriftClip

How your solution addresses the problem statement

Most AIGC detectors are trained and evaluated on clean images, so they can degrade significantly when an image is compressed, resized, cropped, or color-shifted — almost exactly what happens to images that pass through social media.

FusionCLIP addresses this through degradation-consistent paired training (DCPT). During training, each step processes a clean image together with a randomly transformed variant from our 15 transformation types through the same classifier. The training objective penalizes both label error and inconsistency between the two views' features and predictions, explicitly encouraging robustness to common image degradations.

On top of a frozen CLIP ViT-L/14 backbone, we concatenate four engineered signals (DCT frequency-band energy, Laplacian sharpness variance, noise variance, and a CLIP-embedding drift probe) that provide explicit low-level degradation information that CLIP's semantic features alone do not capture.

Our evaluation shows that FusionCLIP maintains strong performance across 15 held-out transformation types. It achieves 0.9699 clean AUC and 0.9651 mean transformed AUC, retaining approximately 99.5% of its clean AUC after transformation, with a clean-to-transformed AUC gap of only 0.0048. This robustness compares favorably with prior AIGC detection work, where performance can degrade substantially after common post-processing and image laundering operations. Our results demonstrate that explicitly training for degradation consistency can substantially reduce this robustness gap.

This suggests that the additional degradation-aware signals provide a measurable, though modest, robustness improvement on top of degradation-consistent training.

Development tools used

VS Code
Git/GitHub
Windows dev environment with CUDA for training
Python
FastAPI + Uvicorn
HTML/CSS/JavaScript

Models or APIs used

CLIP ViT-L/14 as the frozen vision encoder, used purely as a feature extractor.
A custom lightweight MLP classifier trained from scratch on top of the fused CLIP + signal feature vector.
LIQE (via pyiqa), evaluated as an additional signal during ablations (ultimately dropped — it slightly hurt performance, likely redundant with our existing noise/Laplacian signals).

Libraries and frameworks used

PyTorch and torchvision
Hugging Face transformers (CLIP)
pyiqa (LIQE)
OpenCV and SciPy (Laplacian variance, DCT frequency-band signals, noise estimation)
scikit-learn
pandas/NumPy
FastAPI + Uvicorn (web demo backend)

Datasets and assets used

30,000 base images (15,000 real / 15,000 fake) drawn from four sources for generator diversity:

SID-Set (14,000 — primary real-image and social-media-realism source)
CIFAKE (6,000 — native 32×32 images used as an upsampling-robustness stress test)
WildFake (2,500 each across GigaGAN and DALL-E 2) (5000 in total)
GigaGanStyleXL (2500)
Imagen 3 (2500)

Ensures GAN and diffusion family diversity.

Each image is cached with 15 transformed variants (blur, JPEG compression, noise, resize, color jitter, sharpen) for the paired-training scheme — ~450k cached images total.