SynthFlag — from pixels to evidence. An evidence-aware AI-image review signal built for TikTok TechJam 2026.
SynthFlag turns pixels into evidence through a transparent, review-first AI-image detection workflow.
Upload one image to inspect the four-expert ensemble score, with service readiness and limitations shown honestly.
Our decision path: start from the released detector, stress every proposed change, and keep the simplest rule that generalizes.
Four experts—two CLIP and two SigLIP—inspect each image; their AI-class probabilities are averaged into one review signal.
From one RGB image to four expert probabilities: deterministic CLIP/SigLIP paths, exact mean fusion, and provenance-bound outputs.
Two CLIP experts at 224 px and two SigLIP experts at 384 px produce four AI-class probabilities, then SynthFlag averages them equally.
We kept the equal mean, separated score from threshold, rejected fusion that failed domain transfer, and left blocked evidence unavailable.
The 0.28747 operating point improves balanced accuracy and fake recall while lowering precision; ROC-AUC stays 0.8505.
An AIGC image uploaded into our image analyser, being flagged as AI generated.
An AIGC video uploaded into our video analyser, being flagged as AI generated.

Inspiration

Generative AI is becoming increasingly difficult to recognize visually.

A few years ago, AI-generated content often had obvious giveaways: strange hands, distorted faces, broken text, or unnatural textures. Today, those mistakes are disappearing. Once content is compressed, resized, screenshotted, filtered, or reposted through a social-media platform, even more of those clues can disappear.

That creates a particularly important problem for platforms such as TikTok.

TikTok is not primarily an image platform. It is a video platform, where a synthetic clip may contain hundreds or thousands of individual frames. Detecting AI-generated content therefore cannot stop at asking whether a single uploaded image is synthetic.

We wanted to ask a more practical question:

Can we turn AI-image detection into a useful signal for AI-video detection?

That became SynthFlag — from pixels to evidence.

Instead of claiming to definitively prove whether something is "real" or "fake", SynthFlag produces an evidence-based review signal designed to help platforms and moderators identify potentially AI-generated visual content.

What it does

SynthFlag detects AI-generated images and videos.

Image detection

When a user uploads an image, SynthFlag passes it through four independent AI-image detection experts:

2 × CLIP ViT-L/14 experts
2 × SigLIP So400M Patch14-384 experts

Each expert independently estimates the probability that the image is AI-generated.

We apply softmax to each expert's binary output and calculate the exact unweighted average of all four AI-generation scores to produce one final SynthFlag signal.

The released detector uses a calibrated operating threshold of 0.28747 rather than the default 0.5 threshold. On our protected 7,998-image evaluation set, this increased balanced accuracy from 0.7763 to 0.8061 and AI-image recall from 0.5924 to 0.7127, while ROC-AUC remained 0.8505 because the underlying ranking scores did not change.

Video detection — turning a video into evidence

This was especially important to us because TikTok is fundamentally about video.

Instead of requiring an entirely separate video-generation detector, we extended our image detector into a video-analysis pipeline.

When someone uploads a video, SynthFlag:

Keeps the original raw video on the user's device.
Splits the video into representative frames sampled uniformly across its duration.
Extracts eight frames from different points in the video.
Sends only those derived frames to the SynthFlag image detector.
Runs all four detection experts independently on every frame.
Calculates an AI-generation score for each frame.
Summarizes how consistently AI-generated signals occur throughout the video.

In simple terms, we turn:

Video → Frames → AI score per frame → Video-level AI signal

This allows us to reason about questions such as:

"What percentage of the sampled frames in this TikTok appear AI-generated?"

For example, if seven out of eight representative frames strongly exceed our AI-generation threshold, that gives moderators a much stronger signal that the overall video deserves review than relying on one arbitrary screenshot.

The frame-by-frame approach also allows us to preserve a timeline of evidence instead of hiding everything behind one opaque classification.

Our live SynthFlag demo already supports both image and video input and analyzes eight uniformly sampled video frames. Only the derived PNG frames are sent for inference; the original video remains on-device.

Importantly, we do not claim temporal understanding or generator attribution. Frame sampling is a visual review signal rather than proof that an entire video was generated by AI.

How we built it

We built SynthFlag around the released four-expert detector, with clear attribution to the original UESTC research work, and then developed the inference, evaluation, video-analysis, reproducibility, and user-facing layers around it.

1. Four complementary detection experts

Every image or extracted video frame begins as one RGB image and is processed through two separate visual pipelines.

CLIP branch

The image is resized and center-cropped to 224 × 224.

It is then divided into a:

16 × 16 grid = 256 visual patches

before passing through CLIP's transformer representation.

Two independent CLIP-based experts produce predictions.

SigLIP branch

A separate version of the same image is resized to 384 × 384.

SigLIP processes it using:

27 × 27 = 729 patch tokens

and two additional experts independently generate predictions.

This gives us four outputs:

CLIP Expert 1 → CLIP Expert 2 → SigLIP Expert 3 → SigLIP Expert 4

Their AI probabilities are combined using:

Final score = (P1 + P2 + P3 + P4) / 4

We intentionally kept this fusion simple.

During development, we tested a learned fusion layer that produced better pooled development metrics. However, when we tested whether that improvement transferred to unseen datasets, performance fell on important domains such as CIFAKE and WildFake.

Rather than shipping the larger headline number, we rejected the learned fusion and retained the simpler equal-average system because cross-domain generalization mattered more to us than optimizing one benchmark.

2. Extending image detection to TikTok-style video

For video, we built a lightweight browser-side frame extraction pipeline.

The browser selects eight approximately uniformly distributed timestamps throughout the uploaded clip and extracts representative frames.

Those frames become normal SynthFlag image inputs.

This gave us a major architectural advantage:

we did not need to train an entirely separate heavyweight video model to begin detecting AI signals in short-form video.

The same image detector could now examine a TikTok-style video across multiple moments.

It also improves privacy and bandwidth usage because the full video does not need to be uploaded to the inference service. Only the derived frames are scored.

3. Reproducible inference

We also wanted the detector to be usable outside the website.

We therefore built:

a Python inference API;
a resumable command-line inference workflow;
an optional FastAPI service;
checkpoint verification;
SHA-256 checkpoint identity checks;
safe resume for large inference runs;
exclusive inference locking;
CSV result generation;
metadata/provenance output; and
Track 5-compatible JSON output.

For batch evaluation, SynthFlag produces:

predictions.csv
predictions.json
predictions.meta.json

so a prediction can always be connected back to its model, preprocessing configuration, device, and checkpoint identity.

4. Web product

We then turned the detector into something judges and users could actually interact with.

The frontend was built using:

React
TypeScript
Vinext
Vite

The inference stack uses:

Python
PyTorch
Torchvision
Hugging Face Transformers
Pillow
FastAPI

Our development workflow also involved tools such as VS Code and Jupyter/Colab-style Python experimentation environments for model analysis, evaluation, and experimentation.

The final product includes:

an image detector;
a video detector;
a technical architecture walkthrough;
an interactive project journey;
benchmark and threshold explanations;
inference-service health reporting; and
evidence-aware result displays.

Datasets and assets used

Our evaluation work intentionally separated model development, calibration, and protected testing.

For one major evaluation setup, we created a balanced pool of:

10,002 images

5,001 real
5,001 AI-generated

We separated this into:

2,004 calibration/development images
7,998 protected evaluation images

The protected rows remained isolated while we selected our operating threshold and explored other model variants.

Across our experiments, we worked with AI-image domains including datasets such as:

WildFake
CIFAKE
SID_Set

These datasets were particularly useful for evaluating whether improvements generalized across different image distributions rather than simply learning the characteristics of one dataset.

For additional real-image training/evaluation data used in our broader work, we used commercially usable sources including:

40,000 images from CommonCatalog CC-BY
10,000 CC0 images from PD12M

Our broader synthetic-image work also included sources such as:

NTIRE synthetic data
RRDataset synthetic data
UniGenBench-Eval-Images
DRAGON
newly generated synthetic images following our research-inspired generation pipeline

Dataset licensing and redistribution constraints were treated as part of the engineering problem. Public releases intentionally exclude dataset pixels, protected evaluation membership, private predictions, and other material without appropriate redistribution permission.

Challenges we ran into

Making an image detector useful for video

One of our biggest product questions was:

How do we make this relevant to TikTok rather than building another image-only research demo?

Running a large temporal video model would dramatically increase inference cost and deployment complexity.

Instead, we realized that a video is ultimately a sequence of images.

So we created a frame-sampling system that converts a short video into representative visual checkpoints.

This means the model can identify whether AI-generation signals consistently appear throughout the clip instead of judging the entire video from one thumbnail.

It also opens up much richer moderation interfaces in the future: moderators could see where in the timeline AI confidence increases or decreases.

Surviving the real internet

Social-media content rarely reaches a detector in its original form.

TikTok and other platforms resize and compress uploads. Users also screenshot, crop, blur, filter, or re-encode content.

We therefore stress-tested the model against transformations such as:

JPEG compression
resizing
screenshot-like transformations
blur
noise

For example, on our calibration analysis, JPEG Q40 produced an approximately -0.0306 ROC-AUC change, half-resolution downscaling approximately -0.0239, and screenshot-like processing approximately -0.0198.

The important lesson was that robustness is highly dependent on the underlying data distribution.

Preventing ourselves from overfitting the benchmark

Another major challenge was resisting improvements that looked good numerically but did not generalize.

We experimented with a learned head/fusion approach.

Its pooled development performance looked extremely promising.

But once we tested transfer to unseen datasets, the improvement disappeared or reversed.

We rejected it.

That decision became central to SynthFlag's philosophy:

a smaller number that survives unfamiliar data is more useful than a larger number that only survives the benchmark it was developed on.

Deploying a very large detector

The complete four-expert architecture is computationally heavy.

Instead of pretending that a frontend upload automatically meant inference was available, we separated the web application from the durable model service.

SynthFlag explicitly exposes model-service health and refuses to fabricate predictions if the actual checkpoint-backed inference worker is unavailable.

Accomplishments that we're proud of

The thing we are most proud of is that SynthFlag became more than a model checkpoint.

We turned the research detector into an end-to-end system:

Image → patches → four experts → calibrated signal → evidence

and then extended that same idea into:

Video → sampled frames → four experts per frame → frame-level signals → video-level review evidence

That second pipeline is especially exciting to us because it provides a practical path toward identifying AI-generated short-form video on platforms like TikTok without requiring an entirely new video architecture.

We are also proud that we:

preserved a separate 7,998-image protected evaluation set;
achieved 0.8505 ROC-AUC on that protected evaluation;
improved balanced accuracy to 0.8061 using a transparently calibrated threshold;
increased generated-image recall to 0.7127;
rejected a more complicated model when it failed cross-dataset transfer;
built four-expert CLIP + SigLIP inference;
added video frame extraction and frame-by-frame analysis;
kept raw uploaded videos on-device;
built resumable batch inference;
added checkpoint and SHA-256 verification;
created a FastAPI inference layer;
created a polished React/TypeScript product;
built an interactive architecture visualization; and
documented not only what worked, but also experiments that failed or remained incomplete.

What we learned

The biggest thing we learned is that AI detection is not just a classification problem.

It is an evidence problem.

A model returning 0.81 does not prove that an image was created by AI.

Likewise, changing a threshold does not magically make the underlying model better. It simply changes the tradeoff between catching more generated content and incorrectly flagging more real content.

We also learned how important cross-domain evaluation is.

A model can perform extremely well when training and evaluation images share similar distributions while failing badly when images come from another source.

And finally, building the video detector changed how we thought about the problem.

Instead of asking:

"Can we classify this entire video?"

we can ask:

"What does the model see throughout this video's timeline?"

A 20-second TikTok can become eight independent visual observations.

If most of those observations strongly show AI-generation characteristics, the platform receives a much more interpretable moderation signal.

That frame-level evidence is potentially more useful than another unexplained binary label.

What's next for SynthFlag

The next step is making SynthFlag increasingly useful for real social-media moderation.

1. Better video understanding

Our current system analyzes eight representative frames independently.

Next, we want to experiment with adaptive sampling where SynthFlag selects additional frames around:

scene transitions;
sudden confidence changes;
high-AI-confidence regions; or
visually inconsistent segments.

Instead of only saying:

"75% of sampled frames appear AI-generated,"

future versions could show exactly where those signals occur on the TikTok timeline.

2. Temporal AI detection

Our current system intentionally makes no claim about temporal understanding.

Future work could combine frame-level SynthFlag features with lightweight temporal models to detect artifacts involving motion, frame consistency, generated transitions, or temporal coherence.

3. Domain-aware uncertainty

Instead of forcing every image into real versus generated, we want SynthFlag to identify unfamiliar distributions and abstain when evidence is weak.

4. Continue testing social-media transformations

We want to systematically test:

TikTok-style recompression
screen recordings
subtitles and overlays
filters
crops
reuploads
multiple generations of compression

without tuning against our protected test data.

5. Platform-level moderation

Ultimately, we imagine SynthFlag being used not as an automatic censorship system but as part of a moderation pipeline:

Video uploaded → representative frames extracted → SynthFlag scores frames → suspicious videos prioritized → human/platform review

That is why our tagline is:

SynthFlag — from pixels to evidence.

We are not trying to replace provenance, context, or human judgment.

We are trying to give them better evidence.

4. Robustness Evaluation Summary

We evaluated SynthFlag on 15,000 unique images from CIFAKE, SID-Set, and WildFake. Each image was tested twice: once as the original clean image and once after a deterministic composite transformation, producing 30,000 total predictions.

Dataset	Clean ROC-AUC	Transformed ROC-AUC	Δ ROC-AUC	Clean Accuracy	Transformed Accuracy
CIFAKE	0.9816	0.9095	-0.0721	91.98%	82.26%
SID-Set	0.8691	0.8439	-0.0252	78.76%	78.08%
WildFake	0.9467	0.8785	-0.0682	88.24%	77.64%
Macro Average	0.9324	0.8773	-0.0552	—	—

The transformed evaluation applied realistic image corruptions including JPEG compression, Gaussian blur, resizing, Gaussian noise, colour jitter, and cropping. These transformations were applied symmetrically to both real and AI-generated images to simulate the kinds of degradation content can experience after editing, compression, screenshots, and social-media reposting.

Error Analysis Note

False positives, false negatives and TikTok operations. For a creator platform such as TikTok, we consider false positives particularly costly. Incorrectly flagging authentic creator content as AI-generated could wrongly question legitimate work, interrupt distribution or monetization, reduce creator trust, and generate unnecessary moderation reviews and appeals. For that reason, our deployment philosophy prioritizes keeping the false-positive rate low before aggressively reducing false negatives, especially if a detection signal could lead to a consequential platform action. This trade-off is visible in SID-Set: on clean images the detector achieved 99.28% specificity, corresponding to only a 0.72% false-positive rate, but AI recall was substantially lower at 58.24%. In other words, the detector was deliberately conservative: it was more willing to miss some AI-generated content than to incorrectly accuse authentic content. The benchmark also shows why transformed social-media content remains challenging. For example, under composite corruption, WildFake AI recall remained high at 89.72%, but real-image specificity fell to 65.56%, meaning transformations could push legitimate images toward the AI side of the decision boundary. For TikTok, this reinforces why SynthFlag should operate as a review signal rather than an automatic takedown decision: high-confidence detections can help prioritize moderation, while uncertain cases should remain available for additional evidence or human review.