RESONANCE — Solution Summary

RESONANCE is a prototype that detects AI-generated images and stays accurate after real-world post-processing (blur, compression, colour adjustments, cropping, rescaling). It does not retrain or replace the base detector; instead it bolts a small corrective module onto a frozen detector so that the detector keeps working after images have been "handled".

How the solution addresses the problem statement

The problem, restated

The challenge is to tell AI-generated images from authentic ones not just on clean images, but after realistic transformations. Naive detectors collapse under these transformations: a detector trained on pristine images reads local high-frequency details, and blur, JPEG compression or resizing destroy exactly those details.

The technical approach

The pipeline keeps a frozen, clean-trained detector (a DINOv3 vision backbone plus a small classification head) and repairs it from the inside. The key insight is that the image is read twice over the same window:

Spatial branch — the normal read. The window is normalized to a 224px tensor and embedded by the frozen backbone into a feature vector. This is the read that degrades: resizing smears away the high-frequency evidence.
Frequency branch — a second read at native pixel scale. The same window is decomposed into per-block DCT coefficients (8x8 blocks, the JPEG block size), pooled onto a fixed cell grid and log-compressed. Generation traces are a local high-frequency phenomenon, so this basis preserves what the resize destroyed.

Two corrective components live on top of these reads:

GRACE (Gated Residual Adapter for Clean-feature Estimation) — a small label-free module that maps the degraded feature vector back toward what a clean image would have produced. It is trained without any image labels by pulling degraded features onto the detector's own clean features through the frozen head. A per-channel gate decides how much correction to apply, and a "severity head" predicts how corrupted the image is so a mildly degraded image gets a light touch and a heavily degraded one gets more. It is identity at initialization, so any improvement is attributable to what it learned, and it is small (~0.3M parameters against a 21M frozen backbone).
Frequency enricher — trained in a second stage, this module re-reads the DCT tokens and fuses their evidence into the corrected feature using band-gated cross-attention. Two "band experts" read the low- and high-frequency halves of the spectrum separately, because different transformations act differently on the spectrum: blur destroys high frequencies, noise adds them, JPEG moves energy onto block-aligned coefficients. One expert with one gate cannot express all of that; two masked experts can.

Training happens in two stages. Stage 1 trains GRACE label-free against the detector's own clean features (plus a Jacobian-weighted error so capacity is spent on the feature directions that actually move the head's decision, and a severity regression). Stage 2 trains the enricher with image labels, and it is the one place the pipeline uses them. The prediction is the fused logit — the frozen head's read of the fused feature plus a learned scalar times an auxiliary head on the DCT branch's read. Two regularizers make the DCT read earn its place: an auxiliary cross-entropy forces the frequency branch to be label-predictive on its own, and an orthogonality term pushes it into directions the spatial feature does not already span, so the enricher adds forensic content rather than re-stating the spatial read. Both modules start as exact identities, which makes every reported number a measurement of what the module learned rather than of its wiring.

Model sizes

Detector	Trainable modules	Parameters
Baseline (`head(trunk(x))`)	frozen DINOv3 ViT-S/16 trunk + fitted MLP probe head	~22.0M
+ GRACE	baseline + gated residual adapter + severity head	~22.8M
+ GRACE-freq	GRACE + frequency enricher (band-gated cross-attention)	~24.3M

The trunk is frozen in every variant; the corrective modules add ~0.8M and ~1.5M parameters respectively on top of the base detector. The auxiliary head used to train the enricher is a training-only scaffold and does not ship at inference. Counts from scripts/misc/count_params.py.

The evaluation strategy

Degradation grid. Eleven real-world transformations (JPEG re-encode, Gaussian blur, resize, Gaussian noise, brightness/contrast/saturation adjustments, centre crop) are applied at four severity levels: clean, single transform, pairs, and three-to-five transform compositions. Single transformations say why the detector fails; the compositions say how much it fails in deployment.
Shared condition lattice. All detectors are scored on byte-identical degraded images under the same 26 conditions (clean + 19 L1 grid points + L2 and L3 at three replicates each), so a difference between two detectors is never a difference in the random draw. Clean is scored first and fixes the operating threshold (max F1), which is applied unchanged to every degraded condition — so the harness exposes calibration drift that AUC hides and reports a per-condition false-positive / false-negative breakdown at that fixed threshold. Scoring is paired per image, and results can be sliced by generator and by transformation.
Single evaluation arm. All headline runs score crop200, a 200×200 window at native pixel scale — the largest window every benchmark image supplies from its own pixels (the COCO reals ship at exactly 200×200). Within the arm every image has identical dimensions, so the dimension shortcut is 0.5 by construction rather than by normalization. The composed levels are Monte-Carlo estimates, so their AUC carries a bootstrap confidence interval.
Retention metric. The headline number is retention — degraded AUC normalized by the detector's own clean AUC, chance-corrected so a detector at chance scores 0 rather than 0.5: (AUC_deg − 0.5) / (AUC_clean − 0.5). Reported comparisons use the baseline's clean AUC as the shared denominator, because the frequency branch changes both numerator and denominator.
Data integrity. Training and test are disjoint by construction — DALLE and COCO strata are excluded from training — so the detector never sees test generators or reals during training.

Trade-offs and discussion

Every design choice in this solution trades one thing for another; the trades are named explicitly below.

Frozen detector vs. full retraining. We keep the backbone and head fixed and repair their features externally. This trades control over the detector's internals for efficiency: no detector retraining, no image labels in stage 1, and every reported change attributable to the module. The cost is that the repair is bounded by what the frozen head can read — if the head cannot see the corrected directions, the module saturates.
Small adapter vs. capacity. GRACE is ~0.3M parameters against a 21M frozen trunk. This trades raw capacity for evidence: if robustness needed a large module, we would be adding compute rather than demonstrating repair. The risk is an adapter too small to represent the true correction if the mapping is complex.
Identity at init vs. immediate usefulness. Both modules start as exact no-ops (zero-initialized output projections make them identities; the adapter's gate additionally starts near closed, ~0.018). This trades usefulness at step zero for measurability — every number is attributable to what was learned. The cost is harder optimization: the modules must learn to open, and a gate that never opens leaves the repair unused.
Conservative gate vs. correction strength. Starting the gate near closed protects clean and mildly degraded inputs from over-correction (and the false positives that would follow), but it also limits how much correction a heavily degraded image receives, especially early in training.
Severity conditioning vs. reliance on a secondary model. Predicting how far the image has drifted lets the modules apply a light touch to mild damage and heavier correction to severe damage. This trades a small extra module for calibration — and the cost is a dependency: severity is predicted, not given, so errors in the severity head propagate into the correction.
Restoration ceiling vs. a second read. A restorer can at best recover the clean-image score (retention ≤ 1.0); the frequency branch can in principle exceed it. We accept the second read, extra compute, and a supervised second stage in exchange for a higher ceiling.
Band experts vs. a single read. Splitting the spectrum into low- and high-frequency experts lets each specialise — blur destroys high frequencies, noise adds them, JPEG moves them onto block-aligned coefficients — at the cost of added complexity and the assumption that damage types separate cleanly by frequency.
Multi-scale cropping vs. global information. Cropping before the 224px squash removes the resolution shortcut (whole-image dimensions alone score ~0.9997 AUC), so the measurement reflects forensics rather than file metadata. The trade: the model never sees a whole image, and global composition and colour statistics are deliberately discarded.
Generator weighting vs. coverage. Over-weighting recent diffusion and DiT models over GANs (~85–90% of the fakes) matches today's generator frontier — now explicitly augmented with FLUX and Seedream 4.5 — but under-trains on older GAN material; the disjoint held-out benchmark (COCO reals vs DALL-E 3 fakes) tests transfer to unseen generators instead of an in-distribution split.

Development tools used

Python 3.10+ development environment managed with a local virtualenv and pyproject.toml
Code edited in VSCode; command-line-driven experiment scripts (no notebooks required)
pytest for the correctness gates that the pipeline depends on (identity at initialization, reproducible draws, deterministic extraction)
Weights & Biases for optional run tracking and loss/gate logging
git for version control; the paper itself was drafted in Overleaf (LaTeX) with figures authored as Mermaid diagrams

Models or APIs used

DINOv3 ViT-S/16 (via Hugging Face Transformers) — the frozen backbone that embeds images into features
Custom PyTorch modules trained for this work: the MLP probe head, the GRACE gated residual adapter, the severity head, and the frequency enricher (patch-DCT extraction + band-gated cross-attention)
No external generation APIs were used at training time; the data already ships from image generators as part of the dataset

Libraries and frameworks used

PyTorch and torchvision — model definition, training loops, DCT operations
Hugging Face Transformers — loading the frozen DINOv3 backbone
numpy — DCT extraction and feature pipelines
pandas / pyarrow — reading the dataset manifests (Parquet)
scikit-learn — AUC computation and the metadata-only baseline
Pillow — image loading and decoding
matplotlib — plots; tabulate — result tables
pyyaml — configuration files; tqdm — progress bars

Datasets and assets used

WildFake (ModelScope, Apache 2.0) — AIGC-detection corpus used for both training and evaluation:
- Base training: a weighted subset of 49,999 images (14,191 real / 35,808 fake) plus a balanced 10,000-image validation split
- Benchmark: a held-out 13,841-image test set of COCO val2017 reals (4,998) and DALL-E 3 fakes (8,843)
Supplementary SOTA-generator data (Hugging Face):
- FLUX-Reason-6M (Apache 2.0) — 6,000 FLUX images added to training, 2,000 to validation
- seedream-4.5-generated-2k (MIT) — 200 Seedream 4.5 images added to training
- Additional reals — 2,457 (train) / 2,000 (validation) photographs sampled from LAION-5B and ImageNet, disjoint from the manifest's own reals
- Combined corpus: 58,656 training images (16,648 real / 42,008 generated) with a balanced 14,000-image validation split (7,000 / 7,000)
Underlying corpora the reals and fakes come from: LAION-5B and ImageNet (reals); Stable Diffusion, Midjourney, FLUX, Seedream 4.5, and GANs (DF-GAN, GALIP, StyleGAN, GigaGAN, BigGAN, StarGAN) for fakes
Degradation grid — an eleven-transform configuration (YAML) with explicit parameter lists and four severity levels, shared across all experiments
Trained artifacts: probe-head checkpoints, GRACE adapter checkpoints, and enricher checkpoints