Verity

Inspiration

Today anyone can post anything, and it can reach millions of people in minutes. A headline, a screenshot or a forwarded message gets shared thousands of times, and most people never stop to check it: if it's online and everyone is sharing it, it feels true. Research shows the problem is real: false news spreads farther and faster than true news (Vosoughi, Roy & Aral, Science, 2018).

Checking a claim properly means finding sources, reading them and comparing them. That takes time, so almost nobody does it. And AI chatbots don't fix this on their own: they often answer confidently without sources, and sometimes invent quotes.

So we built Verity, a fact-checker that does the checking for you and shows its work: what was checked, where the evidence came from, the exact quotes, and how sure it is.

What it does

Paste any text or an article link. Verity:

Finds the claims worth checking (up to three per check).
Searches the web with Tavily for each claim: queries that support it, refute it, and stay neutral.
Reads the sources and quotes them word for word. Code then checks that every quote really exists on the page. Invented quotes are thrown away and never reach the user.
Asks a jury of two NVIDIA Nemotron models. Nemotron 3.5 Lightning drafts a verdict while Nemotron 3 Ultra judges the same evidence independently, without seeing the draft.
Debates when they disagree. If the models disagree, or both say the evidence is too weak, Verity searches again for what they asked for, and Ultra makes the final call.

Each claim gets TRUE, FALSE, MISLEADING or UNVERIFIED, with a confidence score, a plain-language explanation, the sources with verified quotes, and whether the jury agreed. When both models agree, the final confidence is the average of the two:

$$c_{\text{final}} = \tfrac{1}{2}\left(c_{\text{Lightning}} + c_{\text{Ultra}}\right)$$

Progress streams live while the check runs. A model trace lists every AI call with its model, tokens, time and cost. A typical check takes 10–30 seconds and costs under half a cent per claim.

How we built it

AI: all inference runs on Nebius Token Factory through its OpenAI-compatible API, using only NVIDIA Nemotron models. Nemotron-3_5-Lightning handles the fast steps (claims, search plans, evidence, draft verdict), and Nemotron-3-Ultra-550b-a55b is the independent juror and final judge. We tested every Nemotron model on Token Factory before choosing these two.
Structured output: Token Factory offers function calling but no JSON mode, so every step forces a tool call with a JSON schema. Invalid output goes back to the model once so it can fix it.
Search: Tavily Search with full page text (cached 24 h, max 2 sources per site), and Tavily Extract for article links. Tavily's own AI answers are never used: every verdict comes from Nemotron reading real sources.
Trust: quotes are verified in code against the source text, web content is marked as untrusted in every prompt, and a small list of reliable domains helps ranking.
Stack: Python 3.12, FastAPI, Pydantic v2 and SQLite on the backend; React, TypeScript, Vite and Tailwind CSS on the frontend. 315 backend tests and 39 frontend tests run in GitHub Actions.
Hosting: Vercel, with the FastAPI backend and the Vite frontend as two services in one project. Each check runs inside its own request and streams progress back to the browser.

Challenges we ran into

Reasoning ate the output budget. With reasoning on, Lightning could spend almost all of its output tokens thinking before the tool call, so answers were cut off. We turned reasoning off per step: the evidence step went from 30–46 s to 2–3 s per call. We kept it on only for claim extraction, where it stopped the model from inventing claims.
Quotes broke the JSON. Sources containing double quotes made the model break its JSON while copying them. We now normalize quotes in source text before sending it.
Silently dropped items. Asked to assess 8 sources at once, Lightning sometimes returned only the first. Naming the exact source IDs and count in the prompt fixed it (8 of 8 in 10 of 10 runs), and we retry anything still missing.
Latency spikes and connection errors on some calls: we added retries with backoff inside a fixed time budget.
Serverless hosting: our first design ran checks in the background, and serverless platforms pause that. We redesigned it so each check runs inside its own streaming request.

Accomplishments that we're proud of

Every quote a user sees is verified to exist in its source.
Two different Nemotron models form a real, independent jury, with a debate round when they disagree.
Full transparency: every model call, its cost and its timing are visible on the result page.
A complete, tested, live product at under half a cent per claim.

What we learned

Reasoning models need per-step control: reasoning helps some steps and only adds time and cost to others.
Forced tool calls are a reliable way to get structured output without a JSON mode, as long as the prompt names exactly what it expects.
Checking model output in code (quotes, IDs, counts) catches errors that prompting alone does not.