Inspiration
AI generated images has improved at a drastic rate to the point that nowadays it is hard to tell between AI generated images and real photos. Our own family members especially those of older ages often mistake AI generated images as real photos, this is particularly dangerous due to scams. One of our members uncle had fallen victim to a catfishing scam, if he had this project to show to his uncle at that time, maybe he could have prevented him from falling victim. Hence our overall aim for this project is to help members of society avoid falling into scams involving AI generated images.
What it does
This is an AI image detector, it ultimately aims to look at both semantic and forensic features of a photo to determine how likely an image is AI generated.
How we built it
This project is built through a combination of various guidance found online as well as LLMs such as chatgpt and claude. Training was done with the help of NSCC using their NVDIA A100 gpus.
Challenges we ran into
Time and resource constraints : Some of the datasets used were too large to store on our local machines, and downloading these datasets on the NSCC server took too much time. Training on large datasets, even with the help of the A100 gpus, also took a substantial amount of time, forcing us to cut short training for multiple components of the project.
Shortage of features captured : PEG, blur, resize, noise, colour, and crop pipelines approximate are captured, however features such as social-media codec, screenshot workflow, watermark, filter, or adversarial manipulation are not fully captured. This leads to lesser evidence that the detector can work with and harms its accuracy.
Accomplishments that we're proud of
Rather than using one big model, we ensured robustness through a five stage model. This is because compared to a monolithic model which tries to learn semantic discrimination, frequency-domain forensics, spatial attention, and corruption estimation at the same time and thus receives conflicting gradient signals, the staged approach ensures that each model only focuses on a single optimization. Moreover, it becomes possible to evaluate and retrain each stage in isolation, capturing each problem separately.
Our model doesn't just identify if an image if real or fake, the corruption estimator, predicts the type of augmentation applied and its severity which is used to dynamically re-weight the sementics vs forensics branches of each image in the reliability gate.
What we learned
Consistency losses are cheap and effective. Meaning that enforcing that the model produces the same prediction for a clean image and its corrupted counterpart improves robustness without having to incur high overhead costs, as there are no new parameters or architecture.
One of our core insights was that different augmentations affect semantic and forensic features differently. E.g. jpeg compression destroys frequency domain forensic features but does not greatly affect semantic embeddings. While colour jitter barely affects forensic features. Thus necessitating examining the semantic and forensics separately.
What's next for PixelPerfect
Benchmark on fixed hardware, logging computation resources for documentation. Adopt sharded/streaming storage and make dataset downloads resumable for the WildFake 3.57M image set. Test each individual stage to determine which stage provides the most robustness per compute cost Train the model to detect additional features including but not limited to social-media codec, screenshot workflow, watermark, filter, or adversarial manipulation.
Log in or sign up for Devpost to join the conversation.