Ariadne

Inspiration

In Greek mythology, Ariadne gave Theseus a thread to find his way out of the Labyrinth. In the modern world, online sources have become a labyrinth of their own and we created Ariadne to help unravel and track down AI hallucinations that plague it.

Rather than simply asking whether a document is AI generated or not, we became interested in a harder question: Where did a hallucinated piece of information actually come from and how can we trace it back?

This question is especially relevant with the increasing number of paper mills and AI generated article submissions to conferences.

What It Does

Ariadne takes a source such as a research paper and recursively investigates fake citations and false claims to track down AI hallucinations.

It discovers possible upstream sources, fetches the real documents, and builds an interactive investigation graph showing the claim's lineage (similar to Ancestry.com)!

It then decides where to map these sources in different categories based on their status:

exist - source exists and is not suspicious. These sources are not mapped.
candidate - this source has a high chance of containing fake sources and/or false claims
exist_with_issues - yellow citations either are likely partially AI generated
fake - red citations are likely entirely AI hallucinated

How We Built It

Ariadne uses a recursive multi-source discovery pipeline. This is the general pipeline it follows:

Start: The user inputs a topic or journal article title to search and it finds the corresponding article by scraping the web.
Investigate: Using the GPTZero API, it identifies any sources that are AI hallucinated. It also looks through the text for any false claims or AI generated text. It then uses this information to identify if there are any further candidate sources to explore.
Search and repeat

For every document it reaches, it can search again using:

GPTZero - proposes possible upstream sources
Document-native discovery - extracts links, citations, case names, DOIs, arXiv IDs, docket numbers, and named sources directly from the document
Web search - broadens source discovery

Challenges We Ran Into

We ran into many challenges throughout the process such as:

Issues with discovery recall where in practice, our algorithm would find one source and then stop branching.
Challenges with reaching API rate limits and unpredictable search times.
Displaying the data in a user-friendly manner where it is easy to understand the map of sources.
Finding all the correct and relevant sources without overcrowding the map with unnecessary ones.

Accomplishments That We're Proud Of

Overcoming many of the challenges we ran into in creative ways.
Successfully integrating GPTZero API features.
Scraping for relevant sources that may be potential candidates of AI hallucinations.

Most importantly, we’re proud of working together to see our idea come to life through the project.

What We Learned

Building Ariadne taught us how to combine AI and hallucination analysis with a research pipeline: recursive source discovery, document parsing, and working with GPTZero's detection features. We learned that tracing a claim is a much harder problem than flagging one and deciding what to leave out (like exist sources) and how to categorize what remains as candidates, matters as much as finding the sources in the first place.

What's Next

Ariadne currently traces a hallucination upstream, back up to the sources where hallucinations started. Next, we want to trace it downstream too by seeing how far a hallucinated claim has spread to future papers.

We also want to build a batch analysis capability, so conference organizers and journal editors can upload a whole set of submissions and get a ranked list of which papers most need human review. That turns Ariadne from a single-paper investigation tool into something suitable for peer-review.