FakeProof

In-domain success: Model correctly identifies a real CIFAKE image. 100% confidence occurs as BCE loss pushes probabilities to 100%.
False-positive edge case: High-resolution input saturates the FFT branch, showing the domain gap outside the 32x32 CIFAKE dataset.

Inspiration

AI image generators have crossed a line: they now produce pictures most people can't tell are fake. That's a real problem for any platform where images carry weight — news, profiles, marketplaces. But there's a subtler issue most detectors ignore: images never arrive clean. By the time a suspicious image reaches a moderation system, it's been screenshotted, re-compressed by a social platform, shrunk into a thumbnail, and run through a filter. A detector that scores 95% in the lab but collapses the moment an image is blurred is useless in the wild. We set out to build one that holds up under exactly those conditions.

What it does

Our system takes any image and outputs the probability that it was AI-generated. What makes it different is that it's explicitly built to survive real-world degradation — JPEG re-compression, blur, downscaling, sensor noise, color filters, and cropping. We measure not just clean accuracy, but how well accuracy holds as images get progressively damaged.

How we built it

We combine two complementary views of each image:

DINOv2, a frozen self-supervised vision transformer, gives us rich general-purpose features capturing what an image looks like at a high level — a 768-dimensional vector.
An FFT frequency-domain feature captures the faint, periodic pixel-level fingerprints that generative models leave behind — invisible to the eye but detectable in the frequency spectrum — a 32-dimensional vector.

We turn each image into a feature vector with a frozen DINOv2 backbone, then train a lightweight classifier head on top of it:

image → DINOv2 (frozen, 768-d) → classifier head → P(AI-generated)

Because we freeze DINOv2 rather than retraining it, the whole system is fast to train, cheap to iterate on, and comfortably under the 2-billion-parameter limit.

The key insight: train for damage, don't just test for it. Most robustness work evaluates a clean-trained model on degraded images and reports how much it drops. We went further — we apply the degradation transforms during training, so the classifier learns features that survive blur, compression, and noise from the start. This was the difference between a fragile detector and a genuinely robust one.

Results

The impact was dramatic, and specifically where it mattered. Our clean-trained baseline hit 97% accuracy on pristine images but collapsed to 53–65% — barely better than a coin flip — under heavy blur, 4× downscaling, and strong noise. The augmentation-trained model recovered those failures to 81–85%, a swing of +24 to +32 percentage points on the hardest transforms, for a cost of only about 2% on clean images. Crucially, it stays above 85% accuracy on every single transformation we tested — no remaining condition under which it falls apart.

The most compelling part isn't just that augmentation helped — it's why. The clean-trained model fails asymmetrically: under degradation it doesn't get generally confused, it specifically loses the ability to detect fakes, because blur and compression erase the high-frequency generation artifacts it relies on. Degraded fakes slip through as "real" — the most dangerous failure direction, and exactly what happens most in the wild. Augmentation-aware training forces the model to find cues that survive degradation, closing precisely that gap.

What we learned

Robustness isn't a property you can bolt on at evaluation time — it has to be built into training. And measuring where and why a model fails, not just its headline accuracy, is what turns a number into a genuine insight.

Challenges we ran into

Reproducibility on free infrastructure. We developed on Colab's free tier, where sessions disconnect and GPU quota runs out. We restructured a working-but-fragile notebook into a clean, version-controlled repository with a thin Colab runner, and learned to checkpoint aggressively.
A correctness bug we're glad we caught. Our initial pipeline trained and evaluated on the same test set with no validation split — making its numbers optimistic. We rebuilt it to hold the test set out entirely and split validation from training data only, so our reported numbers are trustworthy.
Version friction. PyTorch 2.6's stricter model-loading defaults and augmentation-library API changes — the ordinary but time-consuming reality of getting a pipeline to run reliably across environments.

What's next

Our clearest limitation is training data: CIFAKE's low-resolution (32×32) images mean the model likely underperforms on high-resolution real-world imagery — a classic domain-generalization gap. Training on higher-resolution, multi-generator datasets is the obvious next step. We'd also explore compositional augmentation (chains of degradations, as happens in reality) and calibrated decision thresholds.