Try It Now: http:joshfcooper.com/telescope

Dislaimer you need to use HTTP not HTTPS for the site to work Code: https://github.com/JoshFCooper415/telescope

Inspiration

After reading up on the latest in AI generated content research, we wanted to both reproduce from scratch, improve on and make easily available the latest advancements in detection for AI generated content. Specifically, we were inspired by the advancements of two recent AI research papers: Binoculars (https://arxiv.org/pdf/2401.12070) for detecting LLM generated content such as the everpresent ChatGPT and Aeroblade (https://arxiv.org/pdf/2401.17879) for detecting latent diffusion image generation models such as Stable Diffusion. When reading these papers, we realized they had problems we could solve.

Binoculars, in its original implementation, uses 2 7 billion parameter models and we believed we could implement a version which only used models that were 20x smaller and faster while still retaining accuracy.
The implementations used in the original paper were out of date with SoTA (State of The Art) LLM's, making them less accurate.
The original implementation of Binoculars had no way to detect AI generated code
The canonical implementations of the papers did not have friendly user interfaces.

To our knowledge, no one else has a platform to detect both AI generated content in text and images to a reasonable level of quality, and we wanted to take the illustrious title of being first to the punch!

What it does

Our project acts as a one-stop shop for all your AI generated content detection needs! We offer 3 modes of usage: Detecting AI generated plain text, images, and a specially tuned version to detect AI generated code. Our method is able to detect inherent traces and signatures that are left behind after image and text generation, which don't change between different image and text generation methods. Whether you are using ChatGPT, Gemini, Claude, or any other generative model, our state of the art technique can spot them even though it hasn't been specifically trained to detect them! In fact our technique will remain robust and future-proof to new and more powerful generative models since they all have the same signature that we can track.

More specifically, for text data our model uses a tool called cross perplexity to calculate how "surprising" a human is to an AI and how "surprising" an AI is to a separate AI. Generally a human is statistically more "surprising" than an AI which in our case means has a higher "perplexity", and this is the signature left behind by almost all AI generated text, which we aim to detect. Conversely, there is no such measure for AI generated images, so we have to use a tool called a variational autoencoder which are baked into many of the most powerful image generation models today. They project images into a language (or latent space) that the model can more easily understand and many AIs simply generate the image through this AI language. However, it turns out that many AI images can be much more easily "put in terms" that the AI can understand than an image that a human creates. Depending on how well the variational autoencoder works on an image, we can use that to help us detect whether an image is likely AI generated.

How we built it

On the backend, we primarily used a combination of python, flask, pytorch, and the hugging face transformers library to make performant and clean code that reproduces cutting edge research in the field from scratch and builds off of that research.

Challenges we ran into

Since we implemented the algorithms from scratch and made our own custom improvements, finding the correct thresholds/ hyperparameters for the exact score that consitituted AI generated content was challenging and highly experimental. In addition, we had to make sure that the interface was simple and easy to use for users unfamiliar with complicated and difficult concepts such as ROC curves to tune our score thresholds for our user's particular application.

Accomplishments that we're proud of

During these few 36 hours, we managed to reproduce 2 academic papers and even make improvements to them using novel techniques to raise accuracy. In addition we made an extremely functional and user-friendly frontend that can easily be used by people who don't understand how our detection algorithm works and how to finetune for ROC, true positive rate, and false positive rate.

What we learned

Building these state of the art detection algorithms from scratch and writing improvements to them strengthened our understanding of how they worked at a fundamental level. In addition, we gained more experience with frontend web development, UI, and UX, attempting to optimize the user experience when using our website even for people who don't fully understand the technology at play.

What's next for Telescope

Although still working well on a vast variety of images, currently our AI generated image detector only works optimally with a specific image size, so in the future, we could figure out some way let the variational autoencoders better handle variable sized image data.

In addition, we would like to scale up this technology to be generally available. Currently, we weren't able to run this on the cloud, since we were afraid of the high costs of AI cloud computing and already had access to a high end consumer graphics card to run inference and testing on locally. In the future, we could scale this technology to be used en-mass using various techniques such as load balancing on cloud gpu clusters if given proper funding.