Real or Fake?

Landing Page
Voting
Results
Summary
Individual Augmentation Robustness
Architecture
Testing with unseen data & later improving the model!
Mobile Landing
Mobile Voting
Mobile Summary

🔗 Live Demo

Play with the model I trained here: Real or Fake? Game Showcase

Download and read the full technical report (PDF)

💡 Inspiration

Most AI-image detectors look great on pristine benchmark files, but break down the second an image is compressed, blurred, or resized on social media.

Built for TikTok TechJam 2026 (Track 5: Robust Detection of AI-Generated Images Under Real-World Transformations), this project asks a simple question: Can a lightweight detector keep ranking synthetic images accurately even after real-world post-processing and edit distortions?

🎯 What It Does

🧠 The Main Deliverable (Core ML Model): A lightweight, open-source detector trained to flag AI-generated images, built to stay accurate across 6 common real-world transformations (blur, noise, cropping, color edits, downscaling, JPEG compression).
🎮 Interactive "Human vs Machine" Web App: A live 10-round game built as a fun proof-of-concept to let people experience the model in action. Players try to spot fake images alongside the AI in real time on identical transformed pixels.
🖥️ Reproducible CLI: A command-line tool that processes any image directory and returns continuous AI confidence scores in clean JSON format.
⚖️ 800-Image Diagnostic Pool: Evaluated on a balanced test set of 400 real and 400 fake images drawn from SID-Set and WildFake benchmarks.

⚙️ How It Works

Architecture: Uses a normalized 512-dimensional frozen OpenCLIP ViT-B/32 embedding paired with a custom-fitted logistic regression head.
Robust Training: Trained on 48,000 image views across CIFAKE, SID-Set, and WildFake. Each original image gets paired with a clean view and an augmented view using one sampled transform (JPEG compression, Gaussian blur, downscaling, noise, color jitter, or cropping).
Deterministic Pixel Sync: Server-side random seeds ensure the backend model scores the exact same transformed pixels served to the player in the browser, keeping ground-truth labels hidden until a guess is submitted.
Removing Frequency Shortcuts: I built and tested a 32-bin radial log-FFT frequency branch. Diagnostic tests showed that while frequency features boosted scores on synthetic benchmarks like CIFAKE, they failed on native-resolution images. I later removed them to ensure real-world reliability.

📊 Key Results

Clean AUC: 0.9578
Condition-Weighted Robust AUC: 0.8966
Workshop Composite Final Score: 0.9272
Validation Promotion Gates Passed:
Held-Out SID Validation AUC: 0.9919
COCO / DALL-E AUC: 0.9029
LAION / DALL-E Matched AUC: 0.9127

🏆 What Makes It Different

Model-First Engineering: Focuses on the competition's core goal with multi-domain dataset balancing, single-corruption training, and disjoint threshold calibration.
Anti-Collapse Promotion Gates: Automatically catches and rejects candidate models that rely on unstable frequency shortcuts before deployment.
Interactive Proof of Concept: Turns abstract robustness metrics into an intuitive game where users can test themselves against the detector on corrupted images.

🛠️ Built With

Machine Learning & Core: Python 3.12, PyTorch 2.9.1 (ROCm 7.2.1), OpenCLIP 3.3.0, scikit-learn, NumPy, Pillow, joblib
Hardware & Compute: Local training on AMD Radeon RX 7900 XTX / Ryzen 7 7800X3D; backend inference hosted on Modal (NVIDIA T4 GPU)
Web App Showcase: FastAPI, React 19, TypeScript, Vite, Cloudflare Pages
Datasets: Kaggle CIFAKE, SID-Set, WildFake

🥊 Challenges & Lessons Learned

Time Constraints: I started late with 12 hours left for the hackathon due to other work, so I switched to the 32x32 CIFake dataset to get a working pipeline and fine-tune models quickly.
Dataset Generalisation: When testing against a baseline, I realised the model struggled with real-world images. This showed me that 32x32 wasn't detailed enough, and I had to move to higher-resolution data.

🚀 What's Next

Test combined transformations (like simultaneous blur and JPEG compression).
Train on more generator-diverse, native-resolution image sources.
Add opt-in aggregate player stats to compare human vs AI accuracy across specific edit types.