InvariantTrace

Inspiration

The challenge was centered around detecting AI-generated images even after they had been modified. What interested us most was the question of how to make that detection remain reliable after those modifications.

Images online rarely stay in their original form. They get compressed, resized, cropped, blurred, screenshotted, edited, and reposted. A detector that performs well only on clean images may not be very useful once those images pass through real-world sharing pipelines.

That led us to focus on one idea: consistency under transformation.

If an image is compressed, resized, blurred, or cropped, the detector should still reach a similar conclusion. That idea became InvariantTrace- a system designed to identify traces of AI generation that remain useful even when the image itself changes.

What it does

InvariantTrace classifies an image as either authentic or AI-generated using a DINOv2-based vision model adapted for binary classification.

Rather than testing the model only on untouched images, we evaluate its performance after applying common real-world transformations, including:

JPEG compression Gaussian blur resizing Gaussian noise color jitter cropping

This lets us measure how well the model retains its prediction when the visual properties of an image change.

We also built a demo application where users can upload an image and view: the predicted class AI probability authentic probability the evaluation context behind the reported model performance

The project also includes a batch inference pipeline that processes image directories and generates predictions in the required JSON format.

How we built it

We built InvariantTrace as a complete machine-learning pipeline rather than just a standalone model.

The system covers dataset preparation, validation, training, inference, robustness evaluation, error analysis, and the final user-facing application.

We used CIFAKE as our primary training and held-out testing dataset.

Our main model uses a pretrained DINOv2 vision backbone. Most of the backbone remains frozen, while a lightweight classification head is trained specifically for authentic-vs-AI classification. This allowed us to take advantage of DINOv2's pretrained visual representations while adapting the model to our task.

A major part of our approach was transformation-aware training.

The model was trained using both clean and transformed versions of images, including blur, compression, resizing, noise, cropping, and color changes. The goal was to encourage the model to learn representations that remain useful despite these perturbations.

We also trained a ResNet18 baseline so that we could compare our robustness-focused approach against a standard vision model.

Challenges we ran into

One of the biggest challenges was realizing that a high test accuracy does not automatically mean strong real-world performance.

A model may perform extremely well when its test data is similar to its training data, yet struggle on images from new sources or distributions. We saw this during testing, where some real-world images received highly confident but incorrect predictions.

This made us much more careful about how we interpreted confidence scores. A prediction of "99% AI-generated" reflects the model's confidence, not absolute certainty.

Dataset size was another practical limitation. We explored larger datasets such as SID_Set, but at roughly 140 GB, training on the full dataset was not realistic within our available hardware, storage, and competition timeline.

We also had to keep the project reproducible while avoiding unnecessary files such as raw datasets, checkpoints, caches, generated outputs, and virtual environments.

Another important decision was keeping the organizer-provided demo dataset completely separate from training, tuning, calibration, threshold selection, and model selection so that our evaluation remained fair.

Accomplishments that we're proud of

Our biggest accomplishment was building a system that performs strongly not only on clean images, but also after image transformations.

Our final DINOv2-based model achieved: 98.54% clean accuracy 95.94% mean transformed-image accuracy approximately 15.77 percentage points improvement in transformed-image accuracy over our baseline

We also analyzed where the detector still struggles. Severe resizing, strong blur, and heavy noise produced some of the largest performance drops. Instead of hiding these results, we included them because understanding failure cases is important when evaluating a detection system.

By the end of the project, InvariantTrace included: a trained DINOv2-based detector a ResNet18 baseline robustness testing across multiple transformations a working demo batch inference with structured JSON output reproducible training and evaluation scripts dataset validation error analysis a clean repository structure

What we learned

The biggest lesson was that robustness is not an optional feature in AI-image detection.

A detector that works on the original image but fails after compression or a screenshot would have limited usefulness in real-world environments.

We also learned that confidence is not the same as correctness. Neural networks can still produce extremely confident predictions on images that fall outside their training distribution.

Working with DINOv2 showed us the power of pretrained vision representations, but also how strongly a model's behavior is still shaped by the dataset used to adapt it.

Most importantly, the evaluation pipeline taught us as much as the training process. Looking at exactly where performance dropped helped us understand what the model had learned and where it remained fragile.

Member Contributions

Arnav Worked primarily on the machine-learning architecture and overall technical direction of InvariantTrace. This included implementing the DINOv2-based model, designing the robustness-focused training approach, building the training and evaluation pipeline, analysing model performance, and helping integrate the trained model into the demo.

Riddhi Focused on dataset preparation, robustness testing, and experimental evaluation. She worked on generating transformed test samples, validating the dataset pipeline, analysing performance across different perturbations, and testing the model on real-world examples.

Austin Worked mainly on the application and inference pipeline. He helped build the user-facing demo, connect the model to the interface, structure the prediction outputs, and implement directory-based inference with JSON export.

Madhu Focused on model evaluation, baseline comparisons, documentation, and project validation. She helped compare DINOv2 against ResNet18, organise experimental results, identify failure cases, verify reported metrics, and structure the final project documentation and repository.

What's next for InvariantTrace

The next major step is testing how well InvariantTrace generalizes beyond CIFAKE.

We want to expand evaluation to more diverse datasets containing real phone photographs, screenshots, social-media reposts, CGI, artwork, edited images, and outputs from newer or previously unseen image generators.

We also want to introduce a stronger uncertainty system. Instead of forcing every image into a confident binary classification, the model could return a "needs review" result when the input is outside its reliable operating range.

Another major direction is generator-held-out and source-held-out evaluation, where entire generators or image sources are excluded from training and only introduced during testing. This would provide a stronger measure of real-world generalization.

Finally, we want to explore combining image-based detection with provenance systems such as Content Credentials. Pixel-level detection alone may never solve every case, so combining multiple independent signals could make AI-image verification significantly more reliable.

InvariantTrace is ultimately about moving beyond clean benchmark accuracy and building AI-image detection that continues to work after an image has actually passed through the internet.