Model Robustness Profile Following AIGIBench Protocol
Anti-Degradation Robustness Across (Clean vs 14 Transformed)

TungTrio AI-Generated Image Detector

Inspiration

Public benchmarks report AI-image detectors exceeding 95% accuracy, but recent large-scale evaluation work (AIGIBench, Li et al., NeurIPS 2025 Datasets & Benchmarks track) shows this is largely an artifact of in-distribution testing: the same detectors' fake-detection accuracy (F.Acc) collapses toward 0% under ordinary perturbations like JPEG re-compression, while their real-image accuracy (R.Acc) stays near 100% — meaning the detector has quietly learned to always predict "real" once its narrow training distribution is perturbed. This project treats that finding as the central design constraint, not an afterthought: every architectural and data-pipeline decision below is aimed at preventing shortcut learning and measuring generalization/robustness honestly, rather than optimizing a single in-distribution accuracy number.

What it does

This project is a binary image classifier that distinguishes authentic photographs from AI-generated (AIGC) images, built around three design priorities:

Generalization to generators and image distributions not seen during training.
Robustness to real-world image degradation (re-compression, resizing, noise).
Calibrated, continuous output (a probability score, not just a hard real/fake label) so downstream users can set their own confidence threshold.

The system combines a self-supervised vision transformer backbone (DINOv2) fine-tuned with LoRA, a fixed-filter high-frequency residual branch (SRM), a data-alignment technique that removes a known shortcut-learning bias (Dual Data Alignment / DDA), and a required degradation-augmentation protocol matched to real-world image redistribution pipelines (JPEG re-encoding, blur, resize, noise, color jitter, cropping).

How we built it

1. Model Architecture

Backbone: DINOv2-ViT-L/14 (or ViT-B/14 for lower-compute runs) with LoRA. DINOv2 was chosen over CLIP-based backbones because its self-supervised training objective preserves low-level structural/textural detail that CLIP's semantic-alignment objective tends to discard — and low-level structural artifacts (not semantic content) are what actually separates real images from generated ones. Only LoRA adapters (rank 8, alpha 16) on the attention query/value projections are trained; the base DINOv2 weights stay frozen. This keeps the trainable parameter count small and training fast, while still letting the model adapt its attention patterns to the detection task.
Auxiliary branch: SRM (Spatial Rich Model) high-pass residual stream. A small bank of fixed (non-trainable) high-pass convolution kernels, drawn from classical steganalysis literature, is applied to the raw image and fed through a lightweight trainable CNN encoder. Because the filters themselves are fixed, the branch cannot "cheat" by re-learning a low-pass filter and discarding high-frequency information — it is architecturally forced to operate on noise-residual statistics (upsampling checkerboard artifacts, generator-specific frequency fingerprints) rather than image semantics. This mirrors published evidence (e.g. AIDE, DFFreq, FreqNet) that fusing a frequency/residual signal with a semantic backbone improves cross-generator generalization over either alone.
Fusion head. The backbone's CLS token embedding and the SRM branch's pooled feature vector are concatenated and passed through a 2-layer MLP with dropout, producing a single logit. A sigmoid at inference time turns this into a continuous 0–1 probability that the image is AI-generated — this continuous score, not a thresholded label, is the model's actual output.

2. Dataset

Primary dataset: SID_Set (saberzl/SID_Set on HuggingFace Hub). A ~300K-image dataset with three classes: real (label 0, sourced from OpenImages V7), full_synthetic (label 1, whole-image AI generation), and tampered (label 2, a real photo with an AI-edited region plus a mask of the edited area). The dataset does not label which specific generator model produced each fake image, so this project does not attempt a per-generator breakdown for SID_Set fakes — instead the pipeline tracks a coarser generator_family/source tag (full_synthetic, dda, tampered) for later error-analysis breakdown.
Target task formulation. By default the classifier is trained on real vs. fully-synthetic only; tampered images are held out separately as a bonus stress test (does a detector trained on whole-image synthesis also catch partial edits?), since tampering is arguably a different task (localized edit vs. whole-image generation) and mixing the two into one "fake" class risks blurring what the model actually learns.
Streaming ingestion. SID_Set is gated on HuggingFace (requires huggingface-cli login and accepting the dataset's terms) and is streamed rather than fully downloaded — the data-preparation script pulls only the number of images per class requested, filtering the stream on the fly, rather than downloading the full ~300K-image dataset before subsampling.
Generalized dataset ingestion. Rather than hard-coding SID_Set's schema, the data-preparation script (scripts/prepare_dataset.py) is built around a common adapter interface (src/data_adapters.py) supporting three source types: any HuggingFace Hub dataset (with a configurable raw-label-to-binary mapping, so it isn't tied to SID_Set's specific label scheme), a local folder tree (flat real//fake/ or nested by generator name), or a CSV manifest. All three produce an identical standardized output layout (data/{train,val,test}/{real,fake}/*.jpg plus a manifest.csv recording each fake's source tag), so the rest of the pipeline is dataset-agnostic.
Split methodology. Images are capped per class (configurable, typically 2,000–3,000 per class per source given compute/time constraints) and split 90/10 or 80/10/10 into train/val/test with a fixed random seed for reproducibility.

3. Data Alignment: Dual Data Alignment (DDA)

Implements Chen et al., "Dual Data Alignment Makes AI-Generated Image Detector Easier Generalizable" (NeurIPS 2025). The motivating problem: a naive way to generate a "hard," well-matched fake counterpart for a real image is to reconstruct it through a generative model's VAE (x̂ = Decoder(Encoder(x))) — but VAE reconstructions retain more high-frequency detail than the real image did, because the real image lost detail to JPEG compression at some point and the VAE reconstruction never went through that compression. A detector trained on such an unaligned pair learns "more high-frequency detail = fake," which is a dataset artifact, not a genuine forgery signal.

DDA corrects this with three steps applied to each real image:

VAE reconstruction (pixel alignment): x̂ = Decoder(Encoder(x)). The paper's own ablation found Stable Diffusion 2.1's VAE performs best for this purpose (versus SD1.5/SDXL VAEs); this project defaults to stabilityai/sd-vae-ft-mse (a publicly accessible standalone VAE checkpoint) since SD2.1's own VAE repository became gated/inaccessible on HuggingFace during development. Images are center-cropped to a multiple of 8 (matching the VAE's downsampling factor) and downsized to a bounded maximum dimension before encoding, to keep GPU memory use predictable regardless of source image resolution.
Frequency alignment: The reconstructed image is re-JPEG-compressed at the same quality factor as the original real image — estimated per-image from the real image's own JPEG quantization table (a quality-factor estimator was implemented and verified to recover the exact original quality on synthetic test cases) — applied with 50% probability during generation, matching the paper's own reported optimum (they found p = 0.5 outperforms always applying it). This is the step that removes the high-frequency shortcut described above.
Pixel mixup: x_mix = r · x_real + (1 − r) · x_freq_aligned, with r ~ Uniform(0, R), blending the aligned fake back toward the real image in pixel space. Default R = 0.5, matching the paper's reported stable range (0.2–0.8).

Paired-batch training: Beyond generating DDA-aligned data, the paper additionally constructs each training batch so that a real image and its own DDA-aligned counterpart land in the same batch together, rather than being randomly separated by ordinary shuffling. This project implements a custom batch sampler (DDAPairedBatchSampler) that discovers real↔DDA pairs purely from a filename convention (dda_<real_image_stem>.jpg), reserves roughly half of each batch's slots for such pairs (configurable), and fills the remainder from ordinary shuffled data — falling back transparently to plain shuffled batching if no DDA-aligned data is present.

Notably, DDA requires no labeled multi-generator fake dataset at all — only real images and a VAE — which is why it is used as an addition to the SID_Set-derived fakes rather than a replacement: it broadens the training distribution with generator-agnostic synthetic examples, complementing SID_Set's specific fakes.

4. Training-Time Augmentation

A separate transform specification (simulating a real-world image redistribution pipeline) is applied during training:

Transform	Parameters	Real-world analog
JPEG Compression	quality = 90, 70, 50, 30	Social-media re-encode, messaging
Gaussian Blur	sigma = 0.5, 1.0, 2.0	Out-of-focus, screenshot smoothing
Resize	scale 0.5× / 0.25×, then upscale back	Thumbnail generation, CDN resize
Gaussian Noise	sigma = 0.02, 0.05, 0.10 (on [0,1]-scaled pixels)	Low-light sensor noise
Color Jitter	brightness/contrast/saturation ±20%	Filter apps, auto-enhance
Center Crop	crop to 80%	Profile-picture cropping, framing

Application methodology, and why: Each training image has a 70% chance of receiving exactly one randomly-chosen transform from this set (never more than one stacked together) — the remaining 30% of images are left clean. This "at most one transform per image" rule is a deliberate choice informed by AIGIBench's finding that combining multiple augmentations on the same training image "offers no clear advantage and can impair performance consistency," particularly for detectors with a frequency-sensitive component (directly relevant here, given the SRM branch).
Evaluation methodology: At evaluation time, the same six transforms are swept individually — 15 conditions total (1 clean baseline + 4 JPEG quality levels + 3 blur levels + 2 resize scales + 3 noise levels + 1 jitter + 1 crop) — never combined, so a robustness drop can be attributed to a specific condition rather than an averaged-away composite. For each condition, R.Acc (accuracy on real images only) and F.Acc (accuracy on fake images only) are reported separately, not just overall accuracy — directly because AIGIBench's central finding is that overall accuracy can look acceptable while a detector has actually collapsed to "always predict real" under perturbation (R.Acc near 100%, F.Acc collapsing toward 0%). The evaluation script additionally flags whichever condition is most damaging to F.Acc specifically, since JPEG compression is reported in the literature as typically the single most damaging condition for fake-detection accuracy.

5. Training Pipeline

Two configuration profiles are maintained for two different compute environments:

Colab free-tier T4 (16GB VRAM): DINOv2-Base/14 (86M params), batch size 16 with 2× gradient accumulation (effective batch 32), fp16 mixed precision, 10 epochs.
Local workstation (RTX 3090, 24GB VRAM): DINOv2-Large/14 (300M params), batch size 32, bf16 mixed precision, 12 epochs.

Both configurations use AdamW (learning rate 1e-4, weight decay 0.01) with a warmup period, and checkpoint the model, optimizer state, and epoch number after every epoch. Training is resumable by default: re-running the same training command automatically continues from the last checkpoint if one exists, rather than restarting — important on free-tier Colab, where a session can disconnect mid-run without warning.

6. Inference Pipeline

A command-line script takes a directory of images and produces a JSON file with one entry per image:

{"image_path": "img.jpg", "pred": 0.93, "label": "AI-generated", "confidence_percent": 93.0}

pred is the model's raw sigmoid output — a continuous probability, not a thresholded binary decision — which is the primary required field. label and confidence_percent are convenience fields derived from pred at a 0.5 threshold, added purely for human readability in a demo context. Test-time augmentation (horizontal-flip averaging) is available as an optional flag, averaging the model's prediction on an image and its horizontal mirror.

7. Robustness Evaluation

Running the full 15-condition transform sweep (see Section 4) against the test set produces a table reporting R.Acc, F.Acc, overall accuracy, and AUC per condition, plus the delta in F.Acc relative to the clean baseline for each condition. This directly instantiates the project's "Robustness Evaluation Summary" deliverable: a compact, condition-by-condition comparison of clean vs. transformed performance, with the R.Acc/F.Acc decomposition specifically included so a superficially acceptable overall-accuracy number cannot mask a collapsed fake-detection rate.

8. Error Analysis

The error-analysis script identifies the most confidently-wrong predictions in both directions — false positives (real images predicted fake) and false negatives (fake images predicted real) — ranked by prediction confidence, so the most surprising failures surface first. It also reports recall broken down by source tag (e.g. full_synthetic vs. dda vs. tampered), which functions as a proxy for cross-generator generalization even though SID_Set itself doesn't label individual generator models.

Trade-off analysis: Three explicit trade-off axes are documented, each tied to a concrete decision made in this pipeline rather than left as abstract commentary:

Robustness vs. clean accuracy: The required-transform augmentation (Section 4) costs some clean-image accuracy in exchange for not collapsing under JPEG/noise at test time — quantified by comparing the Clean-condition row against the worst-case condition in the robustness table.
Generalization vs. specialization: DDA-aligned fakes are generator-agnostic by construction, trading potential peak accuracy on SID_Set's own test distribution for better generalization to generators entirely absent from training — quantified by comparing per-source recall (a lower dda-tagged recall than full_synthetic-tagged recall on this specific test set is the expected, and intended, cost of that choice).
Complexity vs. feasibility: The architecture is already a two-branch fusion (DINOv2+LoRA backbone plus the SRM residual branch); the SRM branch was deliberately kept cheap (a handful of fixed, non-trainable filters, no additional training cost) rather than a second full trainable backbone. DDA's VAE-reconstruction preprocessing step and the paired-batch sampler were both judged worth their added complexity because each maps to a specific, named part of a paper's reported result, rather than being a speculative addition.

9. Development Tools & Environment

Local development / prototyping: Command-line Python environment with bash tooling for scaffolding, syntax-checking, and functional testing of individual components (augmentation transforms, the JPEG quality estimator, the paired-batch sampler's pairing logic) without requiring GPU access for every change.
Training compute: Google Colab (free-tier T4 GPU) as the fallback environment, and a shared Linux GPU workstation (NVIDIA RTX 3090, 24GB VRAM, accessed via SSH) as the primary training environment, using a conda/miniforge environment (aidet) and tmux for disconnect-resilient long-running training jobs.
Version control: Git/GitHub for source control and as the required public code repository deliverable.

10. Libraries & Frameworks

PyTorch — core training/inference framework.
HuggingFace Transformers — DINOv2 backbone loading (AutoModel).
HuggingFace PEFT — LoRA adapter injection (LoraConfig, get_peft_model).
HuggingFace Diffusers — VAE loading for DDA (AutoencoderKL).
HuggingFace Datasets / huggingface_hub — streaming dataset ingestion (SID_Set) with authentication for gated repositories.
torchvision — image transforms (resize, tensor conversion, normalization).
Pillow (PIL) — image I/O, JPEG re-encoding, blur, and the custom JPEG quality-factor estimator built on quantization-table analysis.
NumPy — pixel-level array operations (noise injection, pixel mixup, masking).
scikit-learn — evaluation metrics (accuracy, precision/recall/F1, ROC-AUC).
pandas, PyYAML, tqdm — data handling, configuration, and progress reporting utilities.

Results

Clean Baseline Performance

On the pristine evaluation split (N = 600), the model achieved an overall accuracy of 97.83%, a ROC-AUC of 0.9994, a synthetic detection accuracy (F.Acc) of 99.67%, and an authentic image accuracy (R.Acc) of 96.00%.

Robustness Benchmark Across 14 Redistribution Perturbations

Following the AIGIBench evaluation protocol, each condition was evaluated in isolation against the clean baseline to assess degradation invariance without compound artifact smoothing (Li et al., 2025a):

Condition	R.Acc	F.Acc	Δ F.Acc vs Clean	Accuracy	AUC
Clean Baseline	0.9600	0.9967	+0.0000	0.9783	0.9994
JPEG (q=90)	0.9533	1.0000	+0.0033	0.9767	0.9994
JPEG (q=70)	0.9533	1.0000	+0.0033	0.9767	0.9995
JPEG (q=50)	0.9500	0.9967	+0.0000	0.9733	0.9986
JPEG (q=30)	0.9267	0.9933	-0.0033	0.9600	0.9978
Blur (σ=0.5)	0.9600	1.0000	+0.0033	0.9800	0.9995
Blur (σ=1.0)	0.9433	1.0000	+0.0033	0.9717	0.9995
Blur (σ=2.0)	0.9333	1.0000	+0.0033	0.9667	0.9996
Resize (0.5×)	0.9500	1.0000	+0.0033	0.9750	0.9996
Resize (0.25×)	0.9467	1.0000	+0.0033	0.9733	0.9991
Noise (σ=0.02)	0.9633	0.9867	-0.0100	0.9750	0.9989
Noise (σ=0.05)	0.9267	0.9900	-0.0067	0.9583	0.9970
Noise (σ=0.10)	0.7967	0.9933	-0.0033	0.8950	0.9894
Color Jitter (±20%)	0.9633	0.9933	-0.0033	0.9783	0.9992
Center Crop (80%)	0.9400	1.0000	+0.0033	0.9700	0.9989

Fake Detection Resilience: Across all 14 degradation conditions, Fake Accuracy (F.Acc) remained consistently above 98.67%, successfully preventing the catastrophic collapse toward 0% fake recall commonly observed in baseline detectors under test-time perturbation (Li et al., 2025a).
Compression and Downsampling Invariance: Heavy lossy compression down to JPEG (q = 30) (F.Acc = 99.33%, AUC = 0.9978) and resolution scaling down to 0.25× (F.Acc = 100.00%, AUC = 0.9991) yielded virtually zero loss in synthetic detection power.
Primary Perturbation Bottleneck: Additive Gaussian noise (Noise_sigma0.10) was the most challenging condition, pulling overall accuracy down to 89.50% primarily because authentic images were misclassified as synthetic (R.Acc = 79.67%), even while synthetic accuracy remained intact at 99.33%.

Error Analysis & Misclassification Breakdown

On the 600-sample test set, the detector produced 10 False Positives and only 1 False Negative:

Per-Source Recall: Synthetic detection recall reached 99.67% across all evaluated synthetic images (n = 300).
Single False Negative: The sole missed fake was data/test/fake/synth_002816.jpg (P(Fake) = 0.3339) from the full_synthetic generator family.
Top False Positives: The top failure cases were authentic photos with prominent high-frequency textures and hard edges, led by real_002874.jpg (P(Fake) = 0.9981) and real_002747.jpg (P(Fake) = 0.9794).

Discussion

Eliminating Shortcut Learning via Targeted Alignments

Standard forensic detectors frequently learn high-frequency spectral shortcuts stemming from uncompressed VAE reconstructions. By incorporating Dual Data Alignment (DDA) with dynamic quantization-matched JPEG frequency alignment alongside a bounded training augmentation pipeline (at most one degradation per sample), our model learned invariant structural artifacts rather than brittle dataset quirks. As a result, the model retained over 99.33% synthetic recall under severe redistribution transforms such as JPEG (q = 30) and Gaussian blur (σ = 2.0).

Trade-offs in High-Pass Steganalysis

Integrating a fixed Spatial Rich Model (SRM) filter branch forces the network to focus on microscopic pixel residuals rather than high-level semantic objects. Our evaluation highlights the operational trade-offs of this design:

The Noise Penalty on Specificity: Because the SRM kernels capture high-pass residual differences, severe additive sensor noise (Noise_sigma0.10) mimics synthetic upsampling residuals, depressing authentic image accuracy (R.Acc = 79.67%).
Generalization vs. Specificity Trade-off: The accuracy gap between pristine images (97.83%) and noisy images (89.50%) represents an acceptable operational trade-off to ensure cross-generator transfer and eliminate the default-to-real failure mode.

Architectural Efficiency and Real-World Feasibility

By freezing the DINOv2-ViT backbone and training only lightweight LoRA adapters in tandem with fixed, non-trainable SRM convolution kernels, the architecture eliminates the latency and memory overhead of multi-backbone ensembles. The resulting model delivers sub-40ms inference latency, making it practical for real-time automated moderation workflows.

Challenges & Limitations we ran into

Subsampled Dataset Scale: Trained on a bounded per-class subsample of SID_Set (a few thousand images per class) rather than the full ~300K-image dataset, given hackathon compute and time constraints.
Coarse Generator Labels: SID_Set does not label individual generator models for its fake images, preventing a granular per-generator breakdown (e.g. Midjourney vs. SDXL); DDA's generator-agnostic synthetic pairs partially compensate for this by testing generalization to synthesis methods outside SID_Set's pipeline.
Held-Out Partial Manipulations: Tampered (partially-edited) images were held out of training by default, so performance is optimized for whole-image synthesis rather than localized inpainting.
Substituted VAE Checkpoint: The DDA VAE reconstruction step utilized the standalone stabilityai/sd-vae-ft-mse checkpoint after the official SD2.1 repository became gated on HuggingFace during development.

Accomplishments that we're proud of

Complete End-to-End Pipeline in 72 Hours: Designed, implemented, and benchmarked an entire multi-stream forensic pipeline—spanning streaming data ingestion, VAE reconstruction preprocessing (Chen et al., 2025), LoRA fine-tuning (Hu et al., 2021), automated robustness sweeps (Li et al., 2025a), and interactive deployment—within the 72-hour hackathon timeframe.
Mitigating the "Default-to-Real" Collapse: Successfully defended against the primary failure mode documented in AIGIBench (Li et al., 2025a), retaining over 98.67% Fake-Detection Accuracy (F.Acc) across 14 isolated degradation conditions (including heavy JPEG (q = 30), Gaussian blur (σ = 2.0), and 0.25× downsampling) while maintaining an overall clean AUC of 0.9994.
Engineering Around Gated Infrastructure: When the official Stable Diffusion 2.1 VAE repository became gated on HuggingFace during active development, we rapidly decoupled the data-alignment pipeline, engineered fallback loading logic, and successfully ported Dual Data Alignment (Chen et al., 2025) to the standalone stabilityai/sd-vae-ft-mse autoencoder without sacrificing spectral fidelity or mixup alignment.
Efficient Dual-Stream Fusion: Built an architecture combining macroscopic foundation representations from DINOv2 (Oquab et al., 2023) adapted via LoRA (Hu et al., 2021) and microscopic steganalysis cues via a fixed Spatial Rich Model (SRM) high-pass filter bank, constraining trainable parameters, avoiding the compute overhead of dual full-backbone training, and achieving sub-40ms inference latency.
Production-Ready Explainability & Live Tooling: Delivered both a batch CLI tool (inference.py) outputting continuous probability scores and an interactive Gradio web dashboard equipped with real-time perturbation sliders, latency tracking, and live SRM noise residual heatmaps for interpretable forensic analysis.

What we learned

The Deception of In-Distribution Metrics: High pristine accuracy often masks shortcut learning. Naive synthetic datasets leave uncompressed, high-frequency VAE artifacts that classifiers exploit as shortcuts. Implementing DDA (Chen et al., 2025) demonstrated that rigorous data engineering—specifically dynamic JPEG quantization matching and pixel mixup—is as critical as network architecture in forcing models to learn true generative artifacts.
Controlled Perturbation vs. Destructive Compounding: Unconstrained stacking of heavy augmentations (e.g., compounding JPEG (q = 30) with Gaussian blur and noise) destroys the subtle pixel-to-pixel phase relationships required for forensics, confirming findings from AIGIBench (Li et al., 2025a). Constraining training augmentations to at most one bounded degradation per sample builds invariant representations without corrupting the underlying forensic signal.
Operational Trade-offs in Frequency Steganalysis: While the fixed SRM high-pass filter bank provides strong protection against cross-generator semantic shifts, it introduces an inherent trade-off under sensor noise. Our error analysis revealed that additive Gaussian noise (σ = 0.10) overlaps with the high-pass residual filter band, temporarily lowering real-image specificity (R.Acc = 79.67%). Identifying this operational boundary is vital for deploying forensic tools in uncontrolled environments.
Parameter-Efficient Adaptation for Vision Backbones: Full fine-tuning of vision transformers risks overwriting self-supervised low-level visual representations via catastrophic forgetting (Oquab et al., 2023). Injecting LoRA adapters (Hu et al., 2021) exclusively into the attention query/value projections enabled rapid convergence on synthetic artifacts while preserving DINOv2's spatial feature representations.

What's next for TungTrio

If given more time, we plan to:

Train on the complete SID_Set distribution and incorporate datasets from additional generator families (e.g., Flux, Midjourney v6, Imagen 3).
Implement a broader test-time augmentation (TTA) strategy (multi-crop and multi-scale sweeps rather than horizontal-flip only) to further boost generalization.
Expand fine-tuning and evaluation to include localized tampering and inpainting benchmarks.

References

Chen, R. et al. "Dual Data Alignment Makes AI-Generated Image Detector Easier Generalizable." NeurIPS 2025, arXiv:2505.14359.
Li, Z. et al. "Is Artificial Intelligence Generated Image Detection a Solved Problem?" (AIGIBench). NeurIPS 2025, Datasets & Benchmarks Track, arXiv:2505.12335.
SID_Set dataset: saberzl/SID_Set, HuggingFace Hub.
Oquab, M. et al. "DINOv2: Learning Robust Visual Features without Supervision." TMLR 2024, arXiv:2304.07193.
Hu, E. et al. "LoRA: Low-Rank Adaptation of Large Language Models." ICLR 2022, arXiv:2106.09685.