Detecting AI-Generated Images Using Multi-Scale Attention

In this project, we will reproduce and extend the CIFAKE real-versus-synthetic image classification pipeline originally introduced by Bird & Lotfi (2023).

Comment

Project Summary

In this project, we will reproduce and extend the CIFAKE real-versus-synthetic image classification pipeline originally introduced by Bird & Lotfi (2023). We will first re-implement the baseline CNN classifier and its explainability workflow in PyTorch, replacing the original Grad-CAM visualization with Integrated Gradients for more robust interpretability.

We then introduce a novel artifact-aware architecture that expands the original model’s capabilities by integrating:

  1. Frequency-domain representation (FFT/DCT) to capture generative artifacts
  2. Multi-scale convolutional processing
  3. Attention mechanisms to highlight subtle texture abnormalities typical of synthetic images.

Additionally, we will evaluate three model families: the original CIFAKE CNN baseline, standard pre-trained architectures (ResNet, VGG, ViT), our proposed frequency+attention multi-stream model.

Finally, we will test all models on synthetic images generated by methods not used in the CIFAKE dataset, enabling a cross-generator generalization study.

Methodology

Reproduction (Pytorch):

  • Recreate the CIFAKE dataset loading, preprocessing, and CNN training pipeline.
  • Match or approximate the reported accuracy using equivalent hyperparameters.
  • Replace Grad-CAM with Integrated Gradients.
  • Analyze any discrepancies in performance caused by framework differences (optimizer defaults, initialization, normalization).

Proposed Architecture

We design a dual-stream artifact-aware architecture: Stream A: Spatial Branch

  • CNN backbone (ResNet-18 or custom small CNN).
  • Multi-scale feature extraction using dilated convolutions or pyramid features.

Stream B: Frequency Branch

  • Convert input images to the frequency domain using FFT or DCT.
  • Apply a lightweight CNN to extract frequency patterns (checkerboard textures, periodic noise, high-frequency artifacts).

Fusion Mechanism

  • Concatenate spatial+frequency features.
  • Apply channel + spatial attention (e.g., lightweight self-attention block).

Classifier Head

  • Fully-connected layers leading to a binary real/fake prediction.

Interpretability Plan (Beyond Grad-CAM)

We employ a suite of interpretability techniques to uncover which features drive synthetic-image detection:

  • Integrated Gradients (primary method) – pixel-level attributions.
  • Occlusion Sensitivity – region importance by masking patches.
  • Feature Ablation (frequency masking) – identify which frequency bands matter.

Evaluation Plan

  1. We employ a suite of interpretability techniques to uncover which features drive synthetic-image detection:
  2. Integrated Gradients (primary method) – pixel-level attributions.
  3. Occlusion Sensitivity – region importance by masking patches.
  4. Feature Ablation (frequency masking) – identify which frequency bands matter.
  5. Cross-Generator Generalization (New Synthetic Images) We will evaluate all models on synthetic images generated by newer diffusion or GAN models, such as: Stable Diffusion v2 DALL·E mini / Craiyon, any publicly available GAN fakes or lightly generated samples

Data

  1. CIFAKE Dataset (Primary): The dataset contains 30k real CIFAR-10 images + 30k diffusion-generated synthetic counterparts and is a good source of balanced binary classification.
  2. New Synthetic Dataset (Secondary): We will find small curated set of non-CIFAKE synthetic images for OOD testing, which ensures evaluation beyond the original paper.

Built With

Share this project:

Updates