Project Written Description
How Our Solution Addresses the Problem Statement
Our solution directly tackles the challenge of robust deepfake and AI-generated media detection under real-world degradation. Standard detectors often fail when images undergo post-processing such as social media re-compression, out-of-focus blur, sensor noise, or cropping. To overcome this, our system introduces a two-pronged resilience strategy:
- Degradation-Aware Training: During training, our model is injected with a heavy
Albumentationstransformation pipeline mirroring real-world conditions—including JPEG compression drops (quality 25–95), Gaussian blur ($\sigma$ up to 2.5), low-light sensor noise, color jitter shifts, and regionalCoarseDropoutto simulate local tampering. This prevents the network from relying on clean-lab pixel artifacts. - Test-Time Augmentation (TTA) Inference Engine: At test time, our inference pipeline evaluates multiple transformed variants (scaled, flipped, and original) of each incoming image, averaging their probabilities to stabilize predictions against sudden resolution or framing changes before outputting a calibrated confidence score.
Robustness Evaluation Summary
Our evaluation strategy stress-tested the convnext_tiny prototype against specific real-world augmentations to measure resilience under social media redistribution conditions.
| Evaluation Condition (Real-World Analog) | Overall Accuracy | False Positive Rate (FPR) | Accuracy Drop ($\Delta$) |
|---|---|---|---|
| Clean Data (Baseline) | 95.2% | 1.8% | - |
| Color Jitter (Filter apps, $\pm$20%) | 94.7% | 1.9% | -0.5% |
| Gaussian Noise (Low-light, $\sigma$=0.05) | 93.1% | 2.0% | -2.1% |
| JPEG Compression (Social media, Q=50) | 92.4% | 2.1% | -2.8% |
| Gaussian Blur (Out-of-focus, $\sigma$=1.0) | 91.8% | 2.5% | -3.4% |
| Resize (Thumbnail generation, 0.5x) | 90.5% | 3.2% | -4.7% |
Error Analysis & Trade-Offs
- Representative False Positives: Heavily processed smartphone computational photography (aggressive denoising, HDR blending, face smoothing) and over-filtered social media images occasionally introduced synthetic-looking textures that triggered false alarms.
- Representative False Negatives: Pristine, high-quality AI portraits with realistic skin textures and natural lighting, or images degraded by heavy compression that destroyed subtle generation fingerprints, occasionally slipped past detection.
- Trade-Offs: Using a conservative inference threshold ($0.65$) protects user trust by lowering false positives on authentic photos, but creates a trade-off that increases false negatives for borderline or heavily compressed AI images. Furthermore, global image-level classification trades away fine-grained localization for partial or regional tampering.
Development Tools Used
- IDE & Environment: VS Code, Python 3.10+ virtual environment (
.venv) - Version Control: Git & GitHub
Models & APIs Used
- Backbone Model: ConvNeXt-Tiny (
timm.create_model('convnext_tiny')) – A modern convolutional neural network chosen for its exceptional parameter efficiency (~28M parameters, well under the 2B ceiling), strong spatial awareness, and rapid inference speed suitable for edge deployment.
Libraries & Frameworks Used
- Deep Learning & Computer Vision: PyTorch,
timm(PyTorch Image Models), Albumentations, OpenCV (cv2) - Data Handling & Metrics: Hugging Face
datasets(streaming API), scikit-learn, NumPy, JSON - Backend & Interface: Flask (for local visual dashboard integration)
Datasets & Assets Used
- Primary Training & Validation Source:
saberzl/SID_Setsourced from Hugging Face. The pipeline streams and dynamically balances multi-class source labels into a robust binary classification architecture (Authentic vs. Synthetic/Tampered) to prevent dataset skew.
Log in or sign up for Devpost to join the conversation.