GLowCLIP

Inspiration

AI-generated images do not stay clean after they are shared.

Before reaching a viewer, an image may be compressed by a messaging application, resized for a social-media feed, blurred through repeated uploads, cropped into a screenshot, or altered in color and contrast. These transformations can weaken the local texture and frequency traces used by many AI-image detectors. A model that performs well on clean benchmark images may therefore become unreliable on the transformed version that people actually encounter.

This led us to one central question:

Can an AI-image detector preserve its judgment after an image has been compressed, blurred, resized, cropped, recolored, or otherwise redistributed?

We realized that this problem could not be solved through model architecture alone.

First, the dataset had to be difficult for the right reason. If Real and AI-generated images differ systematically in subject matter, source, format, or image quality, a classifier may achieve a high score by learning shortcuts instead of authenticity.

Second, the detector needed complementary visual evidence. Local forensic traces can reveal generation artifacts, but they are fragile under compression and blur. Global semantic and structural information may be more stable, but it can overlook subtle regional evidence.

We therefore built two connected contributions:

GLowdataset, a balanced, semantically aligned and degradation-aware Real-versus-AIGC benchmark.
GLowCLIP, a detector that adaptively combines global and local evidence and is trained to maintain consistent representations and predictions under degradation.

The name describes the model directly:

Global + Local + Weighted + CLIP

What it does

GLowCLIP is a binary forensic image classifier that distinguishes Real images from AI-generated images.

For each input image, it produces:

A continuous AIGC score
A predicted Real or AIGC label
A validation-selected decision threshold
A diagnostic mean global–local fusion-gate value

GLowCLIP processes every image independently. Deployment images do not require captions, metadata, matched Real/AI pairs, special filenames, class-specific folders, or test-time augmentation. Users can analyze an individual image or recursively process directories containing JPEG, PNG, WebP, BMP, and TIFF files. Each image requires only one forward pass.

We also deployed GLowCLIP as a public Hugging Face Space.

The interactive demo allows users to:

Upload an image.
Analyze its original AIGC score and predicted label.
Select JPEG compression, Gaussian blur, resizing, Gaussian noise, color changes, or center cropping.
Adjust the strength of each degradation.
Preview the transformed image.
Analyze the transformed image again.
Observe how the AIGC score changes after degradation.
Inspect the aspect-ratio-preserving 224 × 224 image passed to the model.
Remove the transformations and return to the original image.

When multiple transformations are selected, the Space applies them in a fixed order:

JPEG → blur → resize → noise → color → crop

This turns our research question into something users can test directly.

GLowCLIP produces a forensic score, not definitive proof of image provenance.

How we built it

Constructing GLowdataset

GLowdataset—stored as glow_dataset in the codebase—combines three complementary sources into one balanced Real-versus-AIGC benchmark.

Source	Matched pairs	Images
Defactify	8,670	17,340
SID_Set	6,787	13,574
DeepGuardDB	894	1,788
Total	16,351	32,702

The dataset contains exactly:

16,351 Real images
16,351 AI-generated images
13,081 training pairs
1,636 validation pairs
1,634 held-out test pairs

The AI-generated side covers eight generator labels:

FLUX: 6,787 pairs
DALL·E 3: 2,032 pairs
Midjourney: 1,734 pairs
Stable Diffusion 2.1: 1,734 pairs
Stable Diffusion 3: 1,734 pairs
SDXL: 1,734 pairs
Imagen: 298 pairs
Stable Diffusion: 298 pairs

Pairing and semantic quality control

GLowdataset is organized around matched Real/AI-generated pairs rather than random class-level concatenation.

Pairs were constructed using native caption relationships or CLIP image similarity:

Defactify pairs satisfy a CLIP similarity threshold of at least 0.75.
SID pairs satisfy a CLIP similarity threshold of at least 0.80.
Other relationships preserve the source datasets’ native caption or pairing information.

The purpose of this pairing is to reduce obvious topic differences between the two labels. A classifier should not succeed merely because the Real class contains people while the AI-generated class contains landscapes.

Our dataset also contains a substantial collection of genuine photographs and AI-generated images from eight modern generator labels. This creates more varied and challenging comparisons than relying on a small Real-image pool, one generator, or only visibly flawed synthetic images.

However, we do not claim that every AI image is photorealistic or visually indistinguishable from a genuine photograph. Semantic similarity also does not guarantee identical composition, viewpoint, lighting, color, or style.

GLowdataset is therefore designed to reduce obvious topic-level shortcuts—not to claim that every possible shortcut has been eliminated.

Pair-aware splitting and degradation

Matched images are divided at the pair level. Both members of a pair always remain in the same training, validation, or test split, reducing the risk of corresponding content appearing on both sides of the evaluation.

We also created controlled modified views for degradation-aware training and robustness evaluation while preserving the original Real or AIGC label.

The transformations cover:

Compression
Resizing
Blur
Noise
Color changes
Cropping

The dataset contribution is the combination of balanced Real–AI pairing, semantically aligned examples, pair-level splitting, multi-source and multi-generator coverage, and degradation-aware evaluation in one reproducible benchmark.

Building GLowCLIP

GLowCLIP is built on the pretrained CLIP ViT-B/16 vision encoder.

CLIP produces two complementary forms of visual information:

A global CLS token representing the image’s overall semantics and structure.
A 14 × 14 patch-token grid retaining information from different image regions.

The global branch projects the final CLS token into a 256-dimensional representation for high-level semantic and structural evidence.

The local branch processes the 196 patch tokens as a spatial grid. A lightweight spatial head preserves regional forensic information, while mean and standard-deviation pooling summarize the strength and spatial variation of the patch features.

A learned 256-channel gate combines the two representations. Unlike fixed averaging, the gate can assign a different balance between global and local evidence for every image and feature channel. It is designed to give the detector flexibility when transformations affect different kinds of evidence differently.

Auxiliary classifiers supervise both branches to discourage the model from relying exclusively on one source of evidence.

Parameter-efficient adaptation

CLIP was originally trained for image–text alignment rather than Real-versus-AIGC forensic classification.

Instead of fully fine-tuning the entire vision encoder, we insert rank-8 LoRA adapters into the query and value projections of the final four CLIP vision layers. The remaining pretrained backbone stays frozen.

This concentrates trainable capacity on task-specific adaptation, the global and local heads, and their fusion mechanism.

Degradation-consistency training

During training, each reference image is paired with an online compound-degraded view carrying the same authenticity label.

Both views pass through the same network. The objective combines:

Fused Real/AIGC classification
Auxiliary global-branch classification
Auxiliary local-branch classification
Cosine feature consistency
Symmetric prediction consistency

These objectives encourage GLowCLIP to maintain similar representations and decisions after realistic image degradation.

The method contribution is not that CLIP, patch tokens, LoRA, or consistency learning are individually new. It is their integration into one detector through:

Per-image, per-channel global–local gating
Spatial patch-level forensic evidence
Global semantic and structural evidence
Degradation-consistency training
Parameter-efficient LoRA adaptation
Single-forward-pass inference

Challenges we ran into

The first challenge was separating genuine forensic learning from dataset shortcuts.

If Real and AI-generated images differ in source, subject, resolution, or format, a classifier may learn those differences instead of authenticity. This made semantic pairing, label balance, and pair-aware splitting central to the project.

The second challenge was that transformations affect different visual evidence in different ways. Compression, blur, and resizing can weaken local traces, while cropping can remove complete image regions. Depending only on global or only on local evidence was therefore risky.

The third challenge was combining heterogeneous source datasets. Their images used different metadata formats, color modes, identifiers, source relationships, and generator labels. We needed a common schema without losing the pairing relationships required for training and evaluation.

Finally, semantic similarity has limits. Even related pairs may differ in composition, viewpoint, lighting, color, or style. We therefore describe GLowdataset as shortcut-reducing rather than shortcut-free.

Accomplishments that we're proud of

We trained GLowCLIP on GLowdataset using random seed 42 and one NVIDIA GeForce RTX 5090 with 32 GB of GPU memory.

The selected epoch-5 checkpoint uses newly initialized LoRA adapters and task-specific heads. The public pretrained CLIP backbone was used as initialization rather than being retrained from random weights.

Held-out modified-image evaluation

The held-out test split contains 3,268 images from 1,634 matched Real/AI-generated pairs. Its operating threshold was selected on the validation set and frozen before testing.

Model	Accuracy	AUROC	AP	F1-score
GLowCLIP (ours)	96.27%	0.9934	0.9940	96.26%
ResNet+LP	87.34%	0.9561	0.9555	88.09%
CLIP+LP	89.60%	0.9614	0.9632	89.59%
NPR (CVPR 2024)	78.82%	0.8836	0.8850	80.61%
VIB (CVPR 2025)	90.59%	0.9693	0.9703	90.76%

These baselines represent four complementary comparison points.

ResNet+LP fine-tunes an ImageNet-pretrained ResNet-18 with a simple binary head. CLIP+LP freezes a pretrained CLIP backbone and learns only a linear classifier. Together, they measure how far conventional pretrained representations can go without GLowCLIP’s global–local fusion.

NPR focuses on local pixel-level resampling residuals. These cues are particularly vulnerable to compression, blur, resizing, and noise, and NPR achieved the lowest result in our degradation-heavy test setting.

VIB adds a variational information bottleneck over frozen CLIP features, providing a robustness-oriented comparison. Unlike GLowCLIP, it does not use an explicit reference–degradation consistency objective.

GLowCLIP achieved the strongest reported result across all four metrics.

Compared with VIB, the strongest reported baseline by AUROC, GLowCLIP improved:

Accuracy by 5.68 percentage points
AUROC by 0.0241
Average precision by 0.0237
F1-score by 5.50 percentage points

Compared with the CLIP linear-probe baseline, GLowCLIP improved:

Accuracy by 6.67 percentage points
AUROC by 0.0320
F1-score by 6.67 percentage points

GLowCLIP additionally achieved:

0.9929 transform-family × degradation-level macro AUROC
96.08% AIGC recall

On a separate same-pair robustness suite covering Defactify and DeepGuardDB, AUROC changed from 0.9890 on clean images to 0.9831 at severity 5, a decrease of approximately 0.006.

These results show that the complete GLowCLIP configuration outperforms the evaluated baselines. Component-level contributions still require systematic ablation studies.

Beyond the metrics, we completed an end-to-end system covering paired dataset construction, degradation-aware training, fixed-threshold evaluation, recursive inference, a downloadable checkpoint, and a public interactive demo.

What we learned

Clean-image performance alone cannot demonstrate real-world robustness. A detector must also be evaluated after transformation and across different sources, generators, and degradation levels.

We learned that dataset design and model design cannot be separated. A sophisticated architecture can still learn shortcuts if the two labels are easy to distinguish for the wrong reason.

Global and local evidence provide different kinds of information. This motivated an adaptive fusion mechanism rather than assigning them the same fixed importance for every image.

We also learned that semantic alignment must be described carefully. A similarity threshold can reduce obviously unrelated content, but it cannot guarantee perfect visual equivalence.

Finally, aggregate performance can hide difficult domains. DeepGuardDB achieved lower performance than the other sources, and the small Imagen subgroup was particularly challenging. All eight evaluated generator labels also appear somewhere in the training data, so the current experiment is not a strict unseen-generator benchmark.

An AIGC score is a forensic signal, not proof. Performance may shift with unseen generators, screenshots, recompression, editing pipelines, camera pipelines, or source-domain changes.

What's next for GLowCLIP

Our next steps include:

Leave-one-generator-out evaluation
Source-held-out and cross-dataset testing
Evaluation on completely unseen generators
Real social-media and messaging-app recompression
Screenshot and multi-stage editing pipelines
Stronger cropping and longer transformation chains
Score calibration for new deployment environments
Systematic ablations of the global branch, local branch, fusion gate, LoRA, and consistency objectives
Detailed false-positive and false-negative analysis

We also plan to extend the Hugging Face Space so users can retain original and transformed scores side by side and visualize how the score changes across multiple degradation levels.

Our goal is not merely to detect the clean image that was originally generated. It is to detect the image people actually encounter after it has been shared, edited, compressed, and seen.