wqsy - TikTok TechJam 2026 Track #5

Dual-stream AI-image detector: CLIP reads what it shows, frequency analysis reads how it was made. Trained to survive compression/cropping, and honestly tested on generators it's never seen.

Comment

How our solution addresses the problem statement

Generative models are getting good enough that visual inspection alone can't reliably separate real photos from AI-generated ones -- and whatever separates them today keeps changing as new generators ship. On top of that, any image posted online gets compressed, resized, cropped, or filtered before anyone else sees it, which can wash out exactly the signal a naive detector relied on. Our submission targets both halves of that problem directly, rather than optimizing for clean-lab accuracy and hoping it holds up.

We built a dual-stream detector: one branch reads what the image depicts (a frozen CLIP ViT-B/32 embedding, the same kind of semantic signal a human uses), the other reads how it was made (a small CNN over an FFT-based frequency residual, capturing the up-sampling/ generator artifacts invisible to the eye but present in the frequency domain). A fusion head combines both into one calibrated probability. Critically, the model is trained through the brief's full transform grid (JPEG re-compression, blur, resize, noise, color jitter, cropping, plus compound/stacked conditions) rather than on clean images alone, so robustness to redistribution is trained in from the start, not patched on afterward.

We evaluated this as a chain of testable claims (V0 spatial-only -> V1

  • frequency -> V2 + transform-aware augmentation), each with its own checkpoint and metrics, so every improvement is measured, not assumed. We also ran a fourth experiment -- a residual-energy-gated frequency stream plus upweighted training on the two conditions where we'd measured a real weakness -- verified it fixed that specific problem, found it cost accuracy on the harder and more central problem the brief itself names first (generalizing to a generator never seen in training), and made the deliberate call to submit the version that doesn't trade away that harder property. That reasoning, plus a concrete FP/FN error analysis, is in the repo's README and this submission's Robustness Evaluation Summary / Error Analysis Note.

Model size and licensing stay within the rules: CLIP ViT-B/32 is ~150M parameters, the frequency CNN and fusion head together are well under 1M, all backbones and datasets are public, and our own code is released under the MIT License (see LICENSE in the repo).

Development tools used

VSCode + terminal, on macOS. Package/environment management via uv. No notebooks (Jupyter/Colab) were used -- the whole pipeline is plain Python scripts under src/, run from the terminal, so every step in the README is copy-pasteable and reproducible outside a notebook.

Models or APIs used

  • CLIP ViT-B/32 (via open-clip-torch, pretrained, frozen) -- the spatial stream's semantic backbone.
  • No closed/paid third-party APIs -- inference is entirely local (CPU/GPU via PyTorch), satisfying the brief's open-source-only and <2B-parameter constraints. Our own frequency CNN and fusion head are trained from scratch and released with the repo.

Libraries and frameworks used

PyTorch + torchvision (model, training loop), open-clip-torch (CLIP backbone), scikit-learn (ROC AUC), SciPy (frequency-residual/FFT processing), OpenCV (opencv-python-headless) and Pillow (image I/O and the transform grid), NumPy, tqdm. Dependency versions are pinned in pyproject.toml / uv.lock; uv sync reproduces the exact environment.

Datasets and assets used

Please refer to submission.zip for the Robustness Evaluation Summary & Error Analysis Note

Built With

Share this project:

Updates