Crossfire

• 

  The logo

What it does

Ask Crossfire a factual question and give it a set of sources. It fetches every page in parallel, extracts what each source actually claims, compares every claim against every other claim, and reports where they disagree — then delivers a verdict with a confidence score and an explanation of how it weighed the evidence.

Most AI search tools blend contradictory sources into one fluent paragraph. That fluency hides the disagreement. Crossfire does the opposite: it makes disagreement the output.

Why the disagreement matters more than the answer

Three findings from building this, all from real runs:

1. Not every disagreement is an error. Asking "what is the population of Tokyo" returns 14.3M and 37M. Both are correct — one is the administrative prefecture, the other the metropolitan area. Crossfire classifies this as same_question_different_scope, a category distinct from genuine contradiction. In our Tokyo run there were 17 scope differences and exactly 1 real contradiction. Collapsing those into a single "sources disagree" signal would be useless; picking one number would be actively wrong.

2. The most authoritative source is not the most discriminating one. On "which Node.js version is Active LTS", the official nodejs.org page labels both v22 and v24 as "LTS" without distinguishing Active from Maintenance. Because it is imprecise, it agrees with everyone and participates in zero contradictions. The source that actually settles the question is the GitHub release schedule table, which marks Active LTS explicitly.

3. Sources that hedge cannot be checked. In the Tokyo run, the Wikipedia "Greater Tokyo Area" page states both figures in a single claim. It participates in 6 pairings and produces 0 contradictions — nothing can disprove it, because it already said both. A source giving one number (citypopulation.de), also in 6 pairings, produced 4 contradictions. Same question, same opponents, opposite behaviour. Comprehensiveness and falsifiability pull against each other, and a verification system has to account for that.

How it works

Four stages, each observable:

1. Fetch — Steel.dev cloud browsers pull every source in parallel. Steel's stateless /v1/scrape endpoint handles most pages; full browser sessions are reserved for pages needing interaction, which keeps us well inside the concurrency ceiling (measured at 10 sessions on the Launch plan).
2. Extract — one claim per page, with its supporting sentence, source tier (primary / news / aggregator), and publication date. Dates matter: most contradictions we found were staleness, not error.
3. Pair — every claim against every other, classified as agree, contradict, unrelated, or same_question_different_scope. Runs concurrently; 28 pairings take 25s instead of 100s.
4. Judge — weighs primary above news above aggregator, prefers recency, and distinguishes "this source is wrong" from "this source is outdated" from "these are different definitions".

The web viewer streams all of this over SSE as it happens: source cards fill in as pages resolve, connectors are drawn between conflicting sources the moment a contradiction is found, and real contradictions (red) are visually separated from scope differences (orange).

What we built for reliability

Failures must be loud. Our worst bug was silent: a pair comparison failed to parse, got downgraded to null, vanished from the conflict list, and never reached the judge. A tool whose entire purpose is finding contradictions quietly dropped one and reported success. The fix has three layers — structured tool-use output so the API guarantees the schema, one automatic retry on failure, and a mandatory visible warning if that retry fails. Both the CLI and the viewer surface incomplete runs rather than presenting a partial result as complete.

HTTP 200 does not mean the page is usable. Of ~54 candidate sources we tested, 21 were unusable: Cloudflare interstitials, soft 404s with a 200 status, JS-rendered skeletons, and pages returning only navigation chrome to datacenter IPs. One URL returned genuine prose — about an entirely different article, because the publisher had recycled the URL. Character count, status code and title all looked fine. is_usable_page() catches the structural cases; the recycled-URL case needed keyword validation.

Retries are visible. The Anthropic SDK silently retries connection errors twice by default. We added an outer retry layer for APIConnectionError specifically — not for auth, rate-limit or schema errors, which are bugs and should surface immediately — and every retry prints to stderr, so a run that struggled never looks like a clean one.

Tested on

The last row is the informative failure. Every source agrees the answer is 8 — the "controversy" is background commentary, not a competing answer. Worse, on genuinely contested topics extraction hedges ("8 under the IAU definition, but up to 150 under others"), and a hedged claim cannot contradict anything. Crossfire's blind spot is precisely the topics everyone already knows are contested. Fixing it requires allowing multiple claims per page, which we scoped out of the weekend.

What's next

The pairing step is a three-way classification over sentence pairs — a task a 140M-parameter cross-encoder can do locally. The pipeline generates exactly the right training data as a by-product: every run produces claim pairs with labels. The plan is to distil that step out of the API and into a fine-tuned NLI model, dropping the dominant cost of the pipeline to near zero. Dataset generation is in progress; results will be added here.

Built solo

One person, arriving three hours late, with no team. Work was split across two agents: Claude Code on the pipeline and prompt engineering, Devin on the viewer and test infrastructure, with a strict file-ownership boundary between them to prevent collisions. That boundary held — until it didn't, in an instructive way: both sides implemented index in the event contract, each self-consistently, with different bases. Any fetch failure would have misaligned every subsequent claim onto the wrong card. Synthetic events never caught it because synthetic events never fail. Only real integration did.

Built With

• anthropic-claude
• asyncio
• css
• fastapi
• html
• javascript
• pytest
• python
• server-sent-events
• steel.dev
• uv
• uvicorn