Initial UI
Final UI, detects fake image
Final UI, identifies real image, robustness analysis included.

regenerators: Robust AI-Generated Image Detection App

How our solution addresses the problem statement

The problem statement encompasses not just detecting AI-generated images, but also detecting them under robust, realistic scenarios. Images can be JPEG-compressed, blurred, colour adjusted — how will we ensure the detection model remains largely accurate after different transformations and redistributions? Our team seeks to answer this.

This project addresses that in three ways:

Robust feature representation. Detection combines a Swin-Tiny vision-transformer RGB branch with a learned Fourier phase-spectrum branch (PhaseEncoder in model.py), fused before classification. Phase-spectrum artefacts left by generative up-sampling/decoding are complementary to RGB texture cues, so the fusion is more resilient to post-processing than an RGB-only classifier.
Parameter-efficient, staged fine-tuning. Rather than fully fine-tuning the backbone (expensive and prone to overfitting on a limited-generator dataset), LoRA adapters (LoRALinear/inject_lora in model.py) are injected into the backbone's attention projections. The model is first trained on CIFAKE, then LoRA-adapted onto a balanced SID-Set subset covering real, fully-synthetic, and tampered images, thus carrying forward general "is this AI-generated" knowledge while cheaply specializing to harder, manipulation-style cases.
Robustness-aware training and inference. Training uses paired clean/degraded views with a consistency loss (losses.py, consistency_weight/consistency_alpha in the configs) so the model's predictions stay stable under JPEG re-compression, blur, resize, noise, colour jitter, and cropping. At inference, inference_policy.py runs deterministic five-view test-time augmentation (clean, JPEG-70, blur-1.0, resize-0.5, crop-0.8) and aggregates logits with a trimmed mean, with the decision threshold selected on a held-out validation split rather than hand-picked. The result (sid_local_lora_best.pt) reaches 0.9474 AUROC on a clean SID holdout and 0.9359 mean AUROC across the full degradation suite (worst condition 0.9049), which is documented and reproducible via reports/metrics/, evaluate.py, and predict.py.

User Interface

Although this is not in scope, we decided to have the detector deployed as a Gradio web app (demo.py/demo_app.py) with an "Analyse image" action and a /robustness lab so the degradation robustness claim can be inspected interactively, not just read from a metrics file.

Robustness Evaluation Summary

The deployed checkpoint (with five-view trimmed-mean TTA, threshold 0.4401) was scored against its clean test images and against 14 additional degradation conditions applied to the same images, reports/metrics/sid_local_lora_tta_test_robustness.json. A representative subset:

Condition	AUROC	Balanced acc.	F1	Precision	Recall	FPR
Clean	0.9474	0.868	0.871	0.850	0.894	0.158
JPEG-70	0.9394	0.870	0.869	0.871	0.868	0.129
JPEG-50	0.9332	0.855	0.844	0.916	0.782	0.072
Blur (σ=1.0)	0.9469	0.876	0.870	0.915	0.829	0.077
Blur (σ=2.0)	0.9049	0.830	0.808	0.923	0.719	0.060
Resize ×0.5	0.9528	0.883	0.884	0.875	0.892	0.127
Resize ×0.25	0.9181	0.844	0.834	0.889	0.785	0.098
Gaussian noise (σ=0.05)	0.9285	0.845	0.849	0.826	0.874	0.184
Crop 80%	0.9462	0.874	0.871	0.893	0.850	0.102
Colour jitter	0.9474	0.867	0.871	0.843	0.901	0.168

Across the full 15-condition suite, mean AUROC is 0.9359 and the worst condition is heavy blur (σ=2.0) at 0.9049 AUROC; the strongest condition is a 2× downscale (resize ×0.5) at 0.9528 AUROC. AUROC degrades gracefully (≤0.05 absolute drop) under everything except aggressive blur, aggressive JPEG (quality 30), and aggressive downscaling (×0.25), the three transforms that most directly erase the high-frequency evidence the phase branch relies on.

Error analysis note

Source class	Clean accuracy	Worst-condition accuracy	Worst condition
Real photos (OpenImages)	0.842	0.811	Gaussian noise (σ=0.10)
Fully synthetic images	0.998	0.928	JPEG quality 30
Locally tampered images	0.790	0.452	Blur (σ=2.0)

False positives (FP) are real images flagged as AI-generated. Noise is the dominant FP driver; blur and aggressive JPEG/resize actually reduce the false-positive rate because the model becomes more conservative once high-frequency detail is destroyed.
False negatives are concentrated almost entirely in tampered images, not fully synthetic ones. Fully synthetic generation is detected reliably (95–99.8% accuracy in every condition tested) because it leaves a strong, global generative signature. Local tampering leaves a much smaller, spatially confined signature that is the first thing degraded by blur, heavy JPEG, or downscaling. Tampered-image accuracy falls from 0.790 (clean) to 0.452 (heavy blur), 0.548 (JPEG-30), and 0.576 (resize ×0.25). This shows the model's weakness: a strong full-synthesis detector and a comparatively weak local-manipulation detector once the image has been re-processed.
Precision/recall trade-off: For clean data, precision 0.850 vs. recall 0.894 — more false accusations of real content than missed fakes, which is the wrong direction since a false "AI-generated" label carries reputational risk. Raising the decision threshold would move precision up and recall down along the curve reported above, but that recall loss might deteriorate with the already-weak tampered class. A precision-first deployment should therefore consider a class-aware threshold or a secondary tamper-specific check rather than a single global cutoff, so that suppressing false positives on real photos doesn't further starve recall on manipulated images.

Development tools used

Local CLI development: Python 3.12.14, with scripts/setup_linux.sh and scripts/setup_windows.ps1 provisioning a matching virtual environment on Linux and Windows, and scripts/verify_environment.py / scripts/check_training_readiness.py gating runs before they start.
Jupyter Notebook (COLAB_TRAINING.md, notebooks/demo.ipynb) for GPU training and for hosting the same Gradio demo, since the primary development machine is CPU-only.
Git for versioning the deployed model checkpoint alongside code.
pytest for the automated test suite (tests/), run before every training or deployment change.
TensorBoard for monitoring training/validation curves during LoRA fine-tuning.

Models or APIs used

Backbone: swin_tiny_patch4_window7_224.ms_in1k, an ImageNet-1k pretrained Swin Transformer Tiny loaded through timm, adapted with LoRA rather than fully fine-tuned.
Custom heads: a convolutional PhaseEncoder over the image's Fourier phase spectrum and a fusion/classification head defined in model.py (AIGCDetector), trained from scratch on top of the frozen-plus-LoRA backbone.
Hugging Face Hub (datasets/huggingface-hub libraries) is used as the API surface for streaming and materializing the SID-Set dataset (scripts/build_sid_subset.py, configs/sid_streaming_lora.yaml) — no external inference API is called; all models run locally/on the training GPU.

Libraries and frameworks used

PyTorch / torchvision: model definition, training loop, and image transforms (installed separately per platform/CUDA build). timm: pretrained Swin Transformer backbone.
Hugging Face datasets and huggingface-hub: streaming/loading SID-Set and caching Hub assets.
albumentations and opencv-python-headless: training-time and benchmark image augmentations (JPEG, blur, resize, noise, colour, crop).
scikit-learn and torchmetrics: AUROC, average precision, F1, balanced accuracy, and threshold selection (metrics.py).
pandas and NumPy: manifest building/merging and numerical processing across scripts/build_manifest.py, scripts/build_wildfake_manifest.py, and data_pipeline.py.
PyYAML: experiment configuration files under configs/.
safetensors: safe checkpoint serialization.
Gradio: the web demo and its /analyse and /robustness API endpoints (demo.py, demo_app.py, demo_inference.py).
matplotlib/seaborn: metrics and robustness-curve plotting.
TensorBoard: training curve logging.
pytest: the automated test suite.

Datasets and assets used

CIFAKE — fast-iteration dataset (32×32, CIFAR-derived, via scripts/download_cifake.sh): initial large-scale real-vs-AI-generated image pretraining set; the CIFAKE-only checkpoint reached 0.9970 AUROC on its held-out test split (configs/full_cifake_lora.yaml).
SID_Set — full-resolution real/synthetic pairs, via scripts/build_sid_subset.py: the primary fine-tuning and evaluation dataset, providing real, fully-synthetic, and locally-tampered images; a balanced 40,000-image subset (configs/sid_local_lora.yaml) produces the deployed checkpoint, evaluated on a stratified 2,000-image model-selection split and a separate 2,000-image holdout (scripts/split_sid_validation.py).
WildFake — an additional multi-generator dataset integrated through scripts/build_wildfake_manifest.py for multi-source robustness experiments (configs/multisource_phase_robust.yaml), preserving the publisher's official train/test split and excluding hackathon-reserved sources (COCO val2017, DALL-E Advanced).
- All datasets are used within the public/research licensing terms; no proprietary or scraped data is included.
Trained model checkpoints (Git LFS-tracked, see ARTIFACTS.md): the deployed checkpoints/sid_local_lora/sid_local_lora_best.pt` (LoRA-adapted Swin-Tiny + phase encoder + fusion head, 6 epochs on the SID subset, initialized from the CIFAKE checkpoint), plus retained legacy checkpoints from earlier CIFAKE-only and rank-4 LoRA experiments.
Reproducible metric reports under reports/metrics/: clean and degradation-suite AUROC/F1/balanced-accuracy results and inference-policy validation, versioned alongside the code that produced them.