Koji's pawns

Compact table that compares performance on clean images versus transformed images.

Inspiration

We were inspired by how quickly AI-generated images can spread online, often after being compressed, reposted, or edited. We wanted to explore a detector that is not only accurate on clean images, but also honest about how reliable its result remains after the transformations images commonly experience on social platforms.

What it does

Koji’s Pawns built RealityCheck, a web app that estimates whether an image is likely AI-generated. Users upload an image and receive an AI-generation confidence score, together with a robustness check that shows whether common real-world edits—such as JPEG compression, blur, resizing, cropping, noise, and colour changes—affect the result.

How we built it

We trained an EfficientNet-B0 image classifier using a carefully selected, balanced subset of the SID_Set dataset. The model distinguishes authentic images from fully AI-generated images, while excluding tampered-image labels to keep the task focused. To test robustness, we built a repeatable transformation pipeline covering 20 scenarios, including JPEG compression, blur, resize, crop, noise, and colour adjustments. We evaluated the model on clean and transformed images using metrics such as ROC-AUC, balanced accuracy, false-positive rate, and false-negative rate. We packaged the detector in a Streamlit web application so users can upload images and view the model’s confidence and robustness results. We also created reproducible scripts, automated tests, and a Kaggle workflow for training and comparing future model versions without replacing the original model automatically.

Challenges we ran into

A major challenge was avoiding misleadingly high results. Image detectors can accidentally learn shortcuts, such as differences in image format, compression, resolution, or source distribution, instead of genuine AI-generation signals. We addressed this by balancing data, applying consistent preprocessing, checking for duplicates, and separating training data from external evaluation data. Robustness was another challenge. Edits like heavy compression, blur, and noise can remove the subtle visual signals that a detector depends on. Rather than hiding this limitation, we measured performance across transformation scenarios and designed the app to present its output as confidence, not proof. We also faced practical constraints: limited training time, limited GPU access, large datasets, and the need to keep downloaded datasets and raw predictions out of the public repository.

Accomplishments that we're proud of

Built a working AI-image detection web app with a clear upload-and-score experience. Created a 20-scenario robustness evaluation instead of reporting only clean-image accuracy. Kept the training, evaluation, and reporting workflow reproducible through automated tests and fixed configurations. Added external evaluation and a separate GenImage v2 training workflow without overwriting the original model. Documented limitations clearly, including the risk of false positives, false negatives, and reduced reliability on unfamiliar generators or heavily transformed images.

What we learned

We learned that AI-image detection is not a simple “real or fake” problem. A model can look strong on data similar to its training set but struggle when images come from new generators or go through real-world reposting pipelines. We also learned the importance of fair evaluation: freezing a model before external testing, tracking dataset provenance, checking for duplicates, using balanced metrics, and reporting limitations honestly. Most importantly, we learned that a confidence score should support human judgement—not replace it.

What's next for Koji's pawns

Next, we want to evaluate RealityCheck on more modern generator families and real platform re-encoding pipelines. We also plan to improve calibration, add an uncertainty or “unable to determine” option for ambiguous cases, and test the model on more diverse real-world image sources. Our long-term goal is to make RealityCheck a practical transparency tool: fast enough for everyday use, clear about its uncertainty, and robust enough to help users think critically before trusting an image online.

Error Analysis

At the fixed 0.50 decision threshold, our robustness-trained model correctly classified 578 of 600 clean SID_Set test images. It produced 14 false positives, where authentic images were labelled as AI-generated, and 8 false negatives, where AI-generated images were labelled as authentic. A representative false positive was an authentic image that received an AI-generated confidence score of 0.832. A representative false negative was an AI-generated image that received a score of only 0.320. These cases demonstrate why RealityCheck presents its result as a confidence estimate rather than definitive proof. Strong Gaussian noise was the most damaging transformation. At the highest tested noise level, false negatives increased from 8 to 65, suggesting that heavy image degradation can remove signals the detector uses to identify AI-generated content. Robustness training improved the worst-case ROC-AUC from 0.986 to 0.990, but slightly reduced clean-image ROC-AUC from 0.997 to 0.996. This represents the central trade-off in our approach: improved resistance to difficult transformations in exchange for a very small reduction in clean-image performance. RealityCheck should therefore be used as a screening aid alongside human judgement, not as the sole proof of an image’s origin.