PreSearch

Inspiration

Conference chairs answer for every paper on their list, and they do not have time to re-check every citation, every number, and every table. Published audits show why that gap matters. A Lancet review of about 2.5 million biomedical papers found references that match no publication rising from about 4 per 10,000 papers in 2023 to 51.3 per 10,000 in late 2025. A separate audit of arXiv, bioRxiv, SSRN, and PubMed Central estimates 146,932 unmatched citations in 2025. At major ML venues in 2025, the share of accepted papers with at least two likely unmatched references was about 1.9% at ICLR, 3.4% at ICML, and 5.1% at NeurIPS. A formatted reference can still point at no work, and a number in the abstract can disagree with the results. PreSearch is the second pair of eyes for that pile: a reason tied to a sentence in the original PDF.

What it does

PreSearch is a quiet desk for the papers a chair is responsible for. They sign in, paste arXiv ids or upload a PDF, and run the list. Each paper walks the same passes: parse, claims, evidence, citations, numbers, tables, dataset, reproduce, verify, critic, stamp. A finding is one of a few checkable problems: a citation the catalogs cannot match, a number in the abstract that never appears in the results, a claimed gain the table does not support, a claim with no passage behind it, or a public table that does not rerun to the stated figure. Summary shows each paper moving through those passes. Chat answers from the stored audit. Click a finding and the original PDF opens on that sentence. Authors have their own page for one paper and its thread. A likeness score never becomes a finding. Passed means those checks found nothing. Failed means there is a countable set of problems, each with a passage.

How we built it

Two programs. The screen is a Next.js app. Sign-in is Supabase, and the browser never holds an API key. The checker is a FastAPI service. It reads the PDF with PyMuPDF, types claims (citation, number, comparison, dataset, semantic) with a page found by searching the PDF, and retrieves supporting and contradicting sentences with a frozen MiniLM encoder from Hugging Face. Citations are asked of Crossref, OpenAlex, and Semantic Scholar. Abstract numbers are compared with the results. A claimed gain is checked against the nearest table. When a paper names a public table, a small allowed calculation (count, sum, mean, and related operations) runs in pandas on a Kaggle CSV. Lya, a Grok model on the xAI API, judges a claim against the evidence rows it was given. Jev, on the OpenRouter Decisions API, runs when Lya’s confidence is under 0.90. Grok writes the chair and author replies and speaks a short briefing through xAI text-to-speech. A finished number check, table formula, or rerun stays final. The page highlight is the same string in the PDF text layer.

Challenges we ran into

The hard part was keeping a model from deciding things code can already check. A judge will happily invent a page, overwrite a subtraction, or treat a catalog timeout as a missing paper. We search the PDF for the span, keep table arithmetic and the pandas rerun ahead of either model, and leave a citation not checked when a catalog errors. Getting embeddings onto Kaggle was its own problem: the Hugging Face client and DNS failed inside the notebook, so the likeness probe uses frozen MiniLM weights shipped with a public dataset. Those Kaggle notebooks also stayed private because the account cannot publish until phone verification. The likeness probe’s holdout AUC is about 0.94, and we still refused to let that score file an issue.

Accomplishments that we're proud of

A chair can paste real arXiv ids, watch the passes, ask the list a question, and open the sentence that failed. On the demo paper, 95.2% is in the abstract and absent from the results, the table computes 89.2 − 84.7 = 4.5 against a claimed 7.8 point gain, and a public Titanic count that claims 317 first-class passengers returns 216. The same desk has an author path, a markdown report and a zip, and a voice briefing whose API key stays on the server. Chat cannot change a stored verdict. The specialist trace shows which check spoke. The recursive playbook beside the desk, typed patches with fitness and replay, is tested, and a patch that drops a fact from a claim is blocked.

What we learned

A finding a chair can use is a sentence, a page, and a reason. Parsers, citation lookups, and table arithmetic have to run before any model. Models are useful in a narrow seat: Grok writes prose, Lya and Jev judge a claim against text they were handed, and neither gets to browse or choose the page. A high similarity-to-generated-text score is a sort key at most. The check that matters is whether the citation exists, whether the number survives the results, and whether the public table reruns.

What's next for PreSearch

Wire the playbook into a live read, so a failed check can teach the next paper only when a second read holds up. Add more public tables than the Titanic CSV. Let a program committee share one list without emailing authors from the app. Keep the rule: PreSearch opens the sentence that did not hold.

Built With

• agent
• cursor
• grok
• jev
• kaggle
• llm
• resreach