Inspiration
AI-generated images are becoming increasingly difficult to distinguish from real photographs. We were interested in a different question: what if detecting AI-generated content was not just about giving someone a single score, but about showing them the evidence behind that score?
This led us to build Veritas, an explainable image-screening tool designed to help users make a faster first-pass assessment of image authenticity.
What We Built
Veritas is a working image-triage prototype that combines multiple signals instead of relying on a single indicator.
The primary signal comes from a pretrained visual AI-image detector. We then add supporting evidence from:
- Benford's Law / pixel-pattern analysis to identify unusual leading-digit patterns in image-derived values
- Camera metadata such as camera make, model, and capture timestamp
- AI-generation software metadata when an image contains a recognizable generator tag
The app combines these signals into an evidence-based assessment and shows users what influenced the result.
Rather than presenting the output as proof that an image is AI-generated or human-made, Veritas is designed as a triage tool that helps determine which images may deserve closer review.
What We Learned
One of the biggest lessons from the weekend was that false positives matter just as much as detection.
We evaluated Veritas on a balanced held-out benchmark of 124 images: 62 real camera images and 62 confirmed AI-generated images.
The external detector alone achieved 66.9% balanced accuracy, but incorrectly flagged 28 of the 62 real images as AI.
With the Veritas provenance blend, balanced accuracy increased to 88.7%, while the number of real images falsely flagged as AI fell from 28 to 0. The system still correctly identified 48 of 62 AI images.
This showed us why an image-screening system needs to consider more than simply maximizing an AI score. A tool intended for human review also needs to minimize unnecessary false alarms and make its evidence understandable.
How We Built It
We built Veritas as a Streamlit application using Python.
The core pipeline:
- Accept an uploaded PNG, JPG/JPEG, HEIC, or HEIF image.
- Run a pretrained ConvNeXt-based AI-image detector.
- Extract available EXIF metadata.
- Calculate a Benford-based image-pattern signal.
- Blend the signals into a final evidence score.
- Present the result through a six-tier assessment, along with an explanation of the evidence that influenced it.
The application runs locally or in a cloud environment, and the external model checkpoint is downloaded and cached when needed.
Challenges
The biggest challenge was deciding how much weight to give each signal.
Benford-based analysis is not decisive by itself, and metadata can be missing, edited, or copied. We therefore designed these signals as supporting evidence rather than standalone proof.
We also had to balance model performance with usability. A technically sophisticated detector is less useful if a reviewer cannot understand why it produced its result.
What's Next
There are several things we would validate with more time.
Improve detection: strengthen the individual visual and forensic signals.
Stress-test performance: evaluate Veritas across more AI generators and common real-world changes such as compression, cropping, editing, and screenshots.
Validate with users: interview organizations that review large volumes of digital images to understand their current verification process and determine whether Veritas solves a meaningful workflow problem.
Our longer-term goal is to move from simply answering "Is this image AI?" toward answering the more useful question:
"What evidence do we have about this image's origin?"
Built With
- benfords-law
- computer
- computer-vision
- convnext
- exif
- law
- machine-learning
- numpy
- pillow
- pillow-heif
- python
- pytorch
- streamlit
- timm
- torchvision
- vision
Log in or sign up for Devpost to join the conversation.