Inspiration

We kept running into the same problem while doing research for school: a report can look completely credible, with authors, citations and confident numbers, and still mislead you. The funder might profit from the result, its sources might all lead back to the same place, and its claims might not be backed by real data. Then we found a newer twist: researchers have started hiding text in papers that only AI tools can see. In 2025, Nikkei Asia found hidden prompts in 17 arXiv preprints telling AI reviewers to give only positive reviews. As more people use AI to read documents for them, documents are starting to target the AI. We wanted a tool that investigates the evidence before you trust it.

What it does

DataDoctor investigates the evidence behind any document. Upload a report or paper (PDF, TXT or Markdown), optionally with the sources it cites, and it:

Screens for manipulation: hidden instructions in tiny fonts, invisible Unicode characters or HTML comments, plus unsafe PDF content like embedded scripts and auto-run actions.
Extracts the essentials: authors, affiliations, funding, claims, citations and methodology.
Traces sources: follows each citation back to its underlying dataset, revealing when "independent" sources share one origin.
Flags concerns: funding conflicts, narrow samples used for broad claims, causal language in survey-only studies, and statistics with no traceable source.
Shows its work: every finding comes with the exact quote, file and page it's based on, an interactive evidence-chain graph, recommended next steps, and a privacy receipt.

It produces a 0–100 score. As a rule of thumb, 90 or above means no major red flags. DataDoctor never accuses anyone; it tells you what to investigate before you believe it.

How we built it

DataDoctor is a Next.js and TypeScript web app with server-side API routes, deployed on Vercel. We wrote a detailed specification and used OpenAI Codex as a coding agent to implement it in milestones, reviewing and testing each step. The pipeline runs in stages: ingest, screen, extract, link, detect, score and validate. PDF text and font sizes come from unpdf (pdf.js), the evidence graph uses React Flow, and data shapes are validated with Zod. An optional AI layer (any OpenAI-compatible model) can help extract claims, but every quote it returns must match the original text or it's discarded. Deterministic code decides every flag and computes the score, so hidden instructions can't change the result. Optional Crossref and OpenAlex lookups check DOIs against public records.

Challenges we ran into

Making AI trustworthy: language models can invent quotes, so we built a verifier that drops anything not found word for word in the document.
Catching invisible text: detecting 1-point hidden text required reading font sizes from PDF text items, not just the text itself.
Not accusing anyone: we added language rules so findings are labeled as facts, potential concerns, interpretations or unknowns, never accusations.
Time: scoping a credible, fully working product into one day meant cutting features and testing edge cases like scanned PDFs, encrypted files and oversized uploads.

Accomplishments that we're proud of

It catches a hidden AI-targeted instruction in a PDF that is invisible to a human reader.
Every finding is backed by a verifiable quote, and the app works fully with zero API keys.
Privacy by design: documents are processed in memory, never stored, and personal data is redacted before any AI call.
An automated test suite and a production smoke test check the whole pipeline end to end.

What we learned

We learned that documents can be an attack surface for AI, that "many citations" doesn't mean "much evidence," and that the safest way to use AI is to let it propose and let deterministic code decide. We also learned how to write precise specifications for an AI coding agent and verify its work.

What's next for DataDoctor

OCR for scanned papers, statistical consistency checks (recomputing reported p-values and catching impossible averages), a browser extension that checks a source in one click, dataset uploads, and mapping an entire literature review at once.