Confused if you are being trolled?

The Problem

AI-generated images have crossed the uncanny valley. But the detection challenge is harder than it appears. By the time an image reaches a social platform, it has likely been JPEG-compressed, resized for thumbnails, blurred through screenshots, or colour-shifted by display captures. These are exactly the transformations that destroy the subtle pixel-level artifacts most AIGC detectors depend on.

So we, team CAPybara started on a quest to answer a harder question: Can we detect AI-generated images using signals that survive real-world redistribution?

Our solution is a hybrid semantic-and-forensic detector that fuses pretrained CLIP visual representations with frequency-domain DCT features, trained across multiple generator families with augmentation-aware robustness built into the pipeline from day one.

How Our Solution Addresses the Problem

Track 5 demands three things simultaneously: accurate detection, generalisation to unseen generators, and robustness under realistic transformations. Our architecture is designed around all three.

Dual-Stream Feature Extraction

For every image, we extract two complementary representations:

Semantic stream — CLIP ViT-L/14 (768-D) A frozen Vision Transformer that captures high-level image structure learned from large-scale image-text pre-training. We chose ViT-L/14 over the more common ViT-B/32 as its 14×14 patch grid gives finer spatial granularity for detecting localised generation artifacts (boundary inconsistencies, texture anomalies, hand distortions) that 32×32 patches average away. At ~300M parameters, it sits comfortably under the 2B competition limit.

Forensic stream — DCT (64-D) The top-left 8×8 block of the Discrete Cosine Transform that is normalised and log-compressed. AI generators leave characteristic frequency-domain fingerprints, especially in how they distribute energy across low-frequency coefficients, that complements CLIP's semantic understanding with a physics-grounded signal.

These two views answer different questions about the same image:

What does the image visually look like? → CLIP
What does its frequency structure reveal? → DCT

The concatenated 832-dimensional vector feeds a lightweight MLP classifier (832→512→128→1) with ReLU activations and dropout regularisation. The entire classifier has ~500K trainable parameters and runs on CPU at inference.

Augmentation-Aware Robustness Training

Robustness was designed into the training pipeline. During feature extraction, every training image is represented both in its original form and with one randomly selected Track 5 transformation applied:

Transform Family	Conditions
JPEG compression	Quality 90, 70, 50, 30
Gaussian blur	σ = 0.5, 1.0, 2.0
Downscale-upscale	0.5×, 0.25×
Gaussian noise	σ = 0.02, 0.05, 0.10
Colour jitter	Brightness / contrast / saturation ±20%
Centre crop	80%

The objective: prevent the detector from depending entirely on artifacts that disappear the moment an image is recompressed, resized, or shared through a messaging app.

The Domain Gap

This is the main story of the project.

On CIFAKE's held-out test set, our classifier achieved 97.4% accuracy and 0.996 ROC-AUC. On WildFake which is the actual evaluation benchmark, the accuracy dropped to 66.6%.

The diagnostic clue was the accuracy-AUC gap: 66.6% accuracy alongside 0.93 AUC. We investigated the raw prediction distribution and discovered it was bimodal, that is, roughly half of DALL-E fakes scored near 1.0 (correctly caught) and the other half scored near 0.0 (confidently wrong), with almost nothing in between. A threshold sweep confirmed this was not a calibration problem. The model had learned Stable Diffusion 1.4's artifacts perfectly but had genuinely never seen anything from the DALL-E family.

Excellent performance within one dataset does not mean the detector has learned a general notion of synthetic imagery. It may instead have learned characteristics of one specific generator, one resolution, or one real-image distribution.

This became the central focus of our development process.

A Critical Labelling Discovery

SID contains three image categories: authentic (0), fully synthetic (1), and tampered/manipulated (2). Our first formulation treated both synthetic and tampered as AIGC. That appeared reasonable, but Track 5 evaluates authentic versus fully AI-generated images.

When we trained with tampered images included, cross-dataset WildFake performance was substantially worse. Excluding the tampered images markedly improved cross-dataset ranking quality with the architecture otherwise unchanged, reaching a WildFake ROC-AUC of approximately 0.93 even before the later dataset expansions.

More training data is not necessarily better training data. : A model benefits most when the labels and training distribution actually correspond to the deployment task.

Closing the Generator Gap

CIFAKE contains a narrow synthetic distribution (Stable Diffusion 1.4 at 32×32). We expanded training with SynthBuster which contains 9,000 images across 9 generator families including DALL-E 2, DALL-E 3, Midjourney v5, Adobe Firefly, multiple Stable Diffusion variants, and GLIDE. Rather than flooding the training set with one generator, SynthBuster introduces a small number of samples from several generators, encouraging the detector to learn broader AIGC characteristics.

Closing the Real-Image Gap

Generator diversity was only half the problem. The real-image distribution matters equally. CIFAKE's authentic images are 32×32 CIFAR-style thumbnails which is nothing like the high-resolution photographs the model encounters at evaluation. We incorporated Tiny-GenImage, which provides both additional generator diversity and higher-quality ImageNet-derived authentic images. This broadened both sides of the classification boundary as a result of which the model now learns what genuine full-resolution photographs look like alongside what a diverse range of generators produce.

Class Balance Fix

Adding SynthBuster (fakes only) skewed the training set to 286K fakes vs 268K reals. This caused false positives to spike, that is, real images were being incorrectly flagged as AI-generated. We implemented deterministic 50/50 undersampling before each training run with no loss-function pos_weight correction (which would have double-corrected). This brought false positives back under control.

The Result

Stage	WildFake Accuracy	ROC-AUC
ViT-B/32 baseline (SID + CIFAKE)	66.6%	0.935
+ SynthBuster + Tiny-GenImage + ViT-L/14	81.0%	0.906

Training Pipeline

Features are precomputed once, then the lightweight MLP trains on the extracted vectors with no expensive CLIP forward passes during training epochs.

Stage 1 — Feature Extraction (~2.7 hours on V100) stream_extract.py processes all five sources in a single pass. SID and Tiny-GenImage stream directly from HuggingFace without disk download. Output: 574,000 feature vectors across 29 compressed .npz shards.

Source	Type	Originals	Vectors	Augmentation
SID	HuggingFace (streamed)	140,000	280,000	×2
CIFAKE train	Local directory	100,000	200,000	×2
CIFAKE test	Held-out validation	20,000	20,000	None
SynthBuster	Local directory	9,000	18,000	×2
Tiny-GenImage	HuggingFace (streamed)	28,000	56,000	×2

Stage 2 — Classifier Training (~9 minutes on V100) train.py loads all shards, undersamples to 50/50 balance, fits z-score normalisation, and trains the MLP for 75 epochs at batch size 512. Internal validation: CIFAKE test (20K images). Best checkpoint: 97.4% accuracy, 0.996 AUC.

Stage 3 — Inference & Robustness Evaluation (~2.5 hours) predict.py loads the frozen checkpoint, scores each image, and outputs the required JSON:

[
  {"image_path": "example/image.jpg", "pred": 0.9134}
]

No external API is required. The entire pipeline runs offline.

Results

Clean WildFake Performance (13,841 images)

Metric	Result
Accuracy	81.02%
ROC-AUC	0.9056
AIGC Precision	92.83%
Authentic Recall	89.60%
AIGC Recall	76.17%

Confusion matrix:

	Predicted Authentic	Predicted AIGC
Actually Authentic	4,478 ✓	520 ✗
Actually AIGC	2,107 ✗	6,736 ✓

When the detector says "AI-generated," it is correct 92.8% of the time.

Robustness Across All 15 Conditions

Condition	Accuracy	ROC-AUC
Clean	81.02%	0.9056
JPEG q=90	81.40%	0.9054
JPEG q=70	79.29%	0.8770
JPEG q=50	76.11%	0.8357
JPEG q=30	74.56%	0.8316
Blur σ=0.5	82.81%	0.9290
Blur σ=1.0	79.42%	0.8734
Blur σ=2.0	59.37%	0.5531
Resize 0.5×	78.57%	0.8626
Resize 0.25×	62.47%	0.6203
Noise σ=0.02	75.49%	0.9035
Noise σ=0.05	72.55%	0.8601
Noise σ=0.10	67.29%	0.7695
Jitter ±20%	79.97%	0.9001
Crop 80%	73.27%	0.9246

Mean per-condition accuracy: 74.91% | Worst condition: Blur σ=2.0 (59.37%)

The model is particularly robust to JPEG compression — the single most common form of real-world degradation — holding above 74% even at quality 30. Mild blur (σ=0.5) actually improves accuracy to 82.8%, likely by suppressing high-frequency noise that sometimes triggers false positives on real images.

Error Analysis

False Negatives — 2,107 DALL-E Fakes Missed

The missed fakes are not random. They concentrate in smooth, photorealistic images of natural scenes — landscapes, food, architecture — where DALL-E Advanced's generation quality is highest. Neither the semantic CLIP features nor the DCT fingerprint deviates sufficiently from the real distribution. The model's boundary was primarily shaped by SD 1.4 and DALL-E 2/3 training examples, and DALL-E Advanced's cleaner frequency signature sits closer to real photographs than its predecessors.

In informal testing, images from entirely unseen generators, particularly Gemini (Imagen 3), score near zero despite being AI-generated. Images with text overlays or screenshot-style borders also fool the model: the re-encoding pipeline strips the frequency artifacts the DCT branch depends on.

False Positives — 520 Real Images Wrongly Flagged

Of 4,998 authentic images, 520 (10.4%) are incorrectly flagged. These cluster around photographs with unusual processing such as heavy HDR tone mapping, artistic filters, aggressive sharpening, or images captured through reflective surfaces which creates frequency patterns resembling AI generation artifacts.

The Asymmetric Risk Trade-off

For a real moderation system, false positives can be more harmful than false negatives — legitimate user content wrongly flagged as synthetic. Our model is comparatively conservative: when it labels an image as AI-generated, it is right 92.8% of the time, at the cost of missing some generated images (76.2% AIGC recall). We consider this trade-off important when thinking beyond benchmark accuracy toward real deployment, where a detector should function as one signal within a broader integrity pipeline rather than making irreversible decisions in isolation.

What Makes Our Approach Different

We did not treat Track 5 as a standard binary classification problem. The system was developed around three interacting forms of generalisation:

Generator generalisation — training across multiple independent image generators rather than memorising one fingerprint.

Real-image generalisation — exposing the authentic class to multiple photographic distributions rather than learning characteristics of one dataset.

Transformation generalisation — embedding all Track 5 transforms into the training pipeline and evaluating each independently.

A reliable AIGC detector should identify signals that generalise across generators, authentic-image sources, and real-world transformations.

Feasibility and Practicality

The architecture deliberately separates expensive feature extraction (CLIP, run once) from lightweight classification (MLP, retrainable in minutes). This provides practical advantages: no foundation-model fine-tuning required, extracted features reusable for rapid experimentation, inference runs entirely offline with no third-party API, datasets stream from HuggingFace without disk download, and the model sits well under the 2B parameter limit.

A realistic deployment could use the detector as one signal within a broader content-integrity pipeline thereby supporting social-media content integrity, misinformation detection, fraud investigation, media provenance systems, journalism and fact-checking with high-confidence detections triggering additional provenance checks or human review.

Limitations and What We Would Improve

Severe transformations remain difficult. Heavy Gaussian blur (σ=2.0) and aggressive downscaling (0.25×) destroy the information both CLIP and DCT rely on. An adaptive gating mechanism that downweights DCT features under detected degradation could mitigate this.

The CLIP embedding is global. Although ViT-L/14 internally processes 14×14 patches, our classifier receives only the pooled representation. Patch-level attention over intermediate ViT tokens could explicitly localise artifacts.

Generator evolution is an open problem. Future generators may exhibit characteristics not represented in any current training dataset. Broader forensic datasets (Community Forensics, AntiFake) and dedicated pretrained AIGC backbones could be ensembled with our CLIP-based detector.

Confidence calibration shifts between domains. Strong ROC-AUC does not guarantee the same optimal threshold across distributions. A threshold calibrated on a held-out DALL-E-family set would improve the accuracy-recall balance.

Frequency-domain augmentation. Applying random spectral perturbations during training could prevent the DCT branch from overfitting to any single generator's frequency signature.

Development Tools

Tool	Purpose
Visual Studio Code	Primary IDE for implementation, debugging, and remote SSH development
Git + GitHub	Version control and team collaboration
CUDA environment	SLURM-managed NVIDIA V100 GPUs for extraction, training, and evaluation

Models

Model	Parameters	Role
CLIP ViT-L/14 (OpenAI, via `open_clip`)	~300M	Frozen visual encoder — 768-D embeddings extracted, no fine-tuning
Custom MLP Classifier	~500K	Trained from scratch on extracted CLIP + DCT features

No external APIs are called during inference.

Libraries and Frameworks

Library	Purpose
PyTorch	Model definition, training, and inference
OpenCLIP	CLIP ViT-L/14 loading and image preprocessing
OpenCV	DCT computation, JPEG simulation, blur, resize, noise, and colour transforms
Pillow	Image I/O and format handling
NumPy	Feature manipulation and compressed shard storage
scikit-learn	Accuracy, ROC-AUC, confusion matrix, and classification metrics
HuggingFace Datasets	Streaming large-scale image datasets without disk download

Datasets

Dataset	Size	Source	Role
SID	140K images	HuggingFace (`saberzl/SID_Set`)	Primary training — diverse real + synthetic; tampered samples excluded
CIFAKE	100K train + 20K test	Kaggle	Secondary training + held-out internal validation
SynthBuster	9K images, 9 generators	Zenodo	Multi-generator diversity: DALL-E 2/3, Midjourney, Firefly, SD, GLIDE
Tiny-GenImage	~56K paired images	HuggingFace (`TheKernel01/Tiny-GenImage`)	High-res reals (ImageNet) + additional generator diversity
WildFake	13,841 images	ModelScope	Evaluation only — never used for training or model selection

What We Learned

The biggest lesson: robust AI-image detection is fundamentally a generalisation problem, not an architecture problem. Our architecture stayed essentially the same from day one. What moved accuracy from 66% to 81% was understanding which generators the model had never seen before and filling those gaps with deliberately chosen public datasets.

The domain gap diagnosis using the accuracy-AUC discrepancy and the bimodal prediction distribution as diagnostic tools was more valuable than any architectural change could have been.

The second lesson: data alignment matters more than data quantity. Removing SID's tampered category improved cross-domain ranking dramatically because those samples represented a fundamentally different forensic task.

The third: both sides of the boundary need diversity. Collecting many generators while training the authentic class on narrow 32×32 thumbnails left the real-image distribution unrepresented. Generalisation requires diversity on both sides.