Persist: Robust AI Image Detection Under Corruption

Inspiration

Most AI-image detectors are benchmarked on clean images and quietly fall apart the moment a real platform touches them — recompression, resizing, a blur filter, a screenshot of a screenshot. A detector that scores 99% AUC on pristine data and collapses under a JPEG re-save isn't actually deployable; it's a leaderboard artifact. Rather than chase invariance to corruption, we asked a different question: what if the model knows it's looking at a corrupted image, and adjusts its confidence accordingly?

What it does

Persist takes a directory of images and outputs a per-image AIGC probability, following the required submission format. It fuses two complementary signal sources — a frozen CLIP ViT-L/14 branch for semantic cues, and a forensic SRM noise-residual branch for low-level pixel statistics CLIP was never trained to see. An auxiliary head estimates the image's corruption severity, and that estimate sets the decision threshold — instead of one fixed cutoff for every image. At inference time, an optional consistency check re-perturbs the image lightly and checks how much the prediction moves, as a secondary signal.

How we built it

image -> [CLIP ViT-L/14 (frozen)] -\ -> [SRM noise-residual branch] -> fusion -> auxiliary distortion/severity head -> severity-calibrated decision

CLIP embeddings are cached once per dataset since CPU-only inference makes live extraction impractical. The auxiliary severity head is trained for free using labels from our own augmentation pipeline. Decision thresholds are calibrated per estimated-severity bucket, with false negatives weighted more heavily than false positives — a deliberate, swept trade-off, not a single untested number.

Challenges we ran into

Our first calibration attempt overcorrected: weighting false negatives 3× pushed false-positive rate to 31.7%, nearly inverting the original imbalance instead of fixing it. We caught this with an explicit confusion-matrix diagnostic and swept the weight down to a genuinely balanced setting (1.5). We also found, through a controlled ablation, that CIFAKE's 32×32 resolution measurably hurts baseline discriminability (a 6.8-point AUC drop when we downsampled otherwise-identical SID_Set images) — though not enough on its own to explain the full corruption-collapse we saw, pointing to resolution and corruption compounding rather than either alone.

Accomplishments that we're proud of

A severity-aware decision process instead of a single fixed threshold, directly validated against our own robustness data. An honest, quantified calibration trade-off rather than an unjustified number. A failure-mode analysis that splits errors into "already wrong on the clean image" (a generalization gap) versus "broken only by corruption" (a robustness gap) — these turned out to be almost inverted between our in-distribution (CIFAKE) and cross-generator (SID_Set) evaluations, which meaningfully changes how you'd actually fix each one.

What we learned

High AUC with a fixed threshold can hide a real deployability problem — accuracy can collapse under corruption even when the underlying ranking is fine. We also learned our severity estimator can misbehave outside the resolution range it was calibrated on: testing on genuinely novel, full-resolution phone photos surfaced this directly, and it's an example of exactly the kind of gap that only shows up once you test beyond your own eval set.

What's next for Persist

Fold our validated consistency-check signal (93.9% standalone separability) formally into the decision logic. Replace the hard severity-bucket boundaries with a smooth, continuous threshold function — we found a case where a borderline severity estimate near a bucket edge flipped a verdict even though the raw prediction barely moved. Train with CLIP embeddings computed live on corrupted images given more GPU budget, and extend cross-generator evaluation further to separate resolution effects from generator-specific effects.