Counterpoint

• 

• 

• 

• 

• 

• 

• 

Inspiration

AI answers high-stakes questions with one confident voice and no way to tell which parts are true. A newer category of "AI debate" apps fixed half of this: several personas argue the decision from different angles. But the personas only state opinions in character — nothing checks whether any of it is real. Users now get five unverified confident answers instead of one, which feels more rigorous while being no more reliable.

The research says this is a real failure mode. Multi-agent debate does improve accuracy, but most work evaluates only the final answer and never the quality of the intermediate arguments. Work on persuasion in multi-agent debates shows rhetorical style can beat correctness when one agent is right and one is wrong.

Debate without verification optimises for persuasiveness, not truth. We wanted to see what happens when you bolt a fact-checker onto the debate itself.

What it does

You describe a real decision. Counterpoint restates it neutrally and asks the three questions only you can answer.

Three advisors — an Advocate, a Skeptic and a Pragmatist — then argue it over two rounds. Every factual claim they make is pulled out of the transcript, classified, and searched against live web sources while the next advisor is already speaking. Claims light up grey while being checked, then flip to their verdict colour in the text itself.

A claim that fails verification forces its author to retract it in round two. Verification changes the debate rather than annotating it.

It closes with a recommendation, concrete next steps, and an honest ledger: "this rests on 6 verified and 12 unverified claims."

Five verdicts, not true or false

Verdict	Meaning
Verified	Independent sources support it. Link and quote shown.
Disputed	Sources conflict with it, or contradict it outright.
Unsupported	Checkable in principle, but no evidence found. Not the same as false.
Depends on you	Truth depends on facts only you know.
Opinion	A value judgment. Not the kind of thing that can be checked.

The two in the middle are the point. "No evidence found" is not the same as false, and "depends on facts only you know" is not a failure of the check — it is the question you still have to answer. Collapsing those is what makes other tools feel more certain than they are.

Two features fall out of this for free:

• The Known vs Assumed meter. One bar showing what the recommendation actually rests on. Nobody else shows that number.
• Open questions come free. Every "depends on you" claim is already an unanswered question, tied to the argument that needs it.

An advisor scoreboard

Because every claim is attributed and checked, we can show which advisor was actually reliable — computed from the claims already collected, at no extra cost. In our recorded startup-offer run the Advocate finishes 0 verified, 3 refuted, 3 retracted while the Pragmatist goes 3 verified, 0 refuted and is marked most reliable.

How we built it

Next.js 16, React 19, TypeScript, Tailwind v4 and Framer Motion. Debate and synthesis run on Groq's gpt-oss-120b; claim extraction and fact-checking run on gpt-oss-20b; verification is against Tavily web search. A full run costs about 13 model calls and 13 web searches in roughly 55 seconds, streamed to the browser as NDJSON events.

Three decisions shaped the build:

One call per turn. Each advisor returns its argument and the claims inside it in a single response, indexed by sentence. Claim indices therefore map exactly onto the sentences that get rendered — no fuzzy string matching to decide which words to highlight.

Checks overlap generation. A turn's claims are verified while the next advisor is already speaking. Only the round boundary waits, because round two needs the verdicts in order to force retractions. This took a run from three minutes to under one.

Record and replay. Completed runs are saved as event logs and replayed with zero API calls. This made UI iteration free, and gives a live demo a fallback that cannot fail from a dead network or an exhausted rate limit.

Challenges we ran into

Advisors returned nothing at all. gpt-oss models spend around 1,300 tokens on hidden reasoning before emitting any content. Our token budget was consumed entirely by reasoning, so every advisor returned an empty string. Fixed by capping reasoning effort and sizing budgets for what the model actually emits.

Rate limits. Groq's free tier allows 8,000 tokens per minute per model, and a debate burns far more than that in bursts — runs died mid-way with 429s. Fixed by moving fact-checking onto a different model so the two draw on separate quota buckets, capping concurrency, and waiting out the exact cooldown the API reports.

Truncated JSON. Groq rejects an incomplete JSON response outright rather than returning the fragment, so one long-winded advisor killed an entire debate with a 400. Now retried automatically with double the allowance.

Three advisors agreeing with each other. Early runs had all three citing the same three numbers, which is useless as a debate. Fixed by passing every claim already on the table into each prompt with an instruction to dispute it or find different ground.

Verdicts with no visible source. The judge returned "verified" but the source index didn't map, so claims showed as verified with nothing behind them — the single worst failure this product could have. Now recovered by quote matching before falling back.

What we learned

Prompting for checkable claims is a different skill from prompting for good answers. Our first advisors produced confident, fluent, entirely unverifiable prose — invented precision about the user's own situation, and statistics attributed to named analyst firms that do not appear to exist. Getting them to state plain, lookup-able facts about the world took more prompt work than the debate structure itself.

We also learned the honest answer is often uncomfortable. A typical run comes back around 30% verified. That number is the product.

Limitations we are honest about

Counterpoint does not determine truth. It separates what has been checked from what has not, and makes that visible before you commit. A "verified" tag means sources support the claim, not that it is certainly true, and quality is bounded by what web search surfaces. Decisions resting mostly on private facts will correctly show a low "known" percentage — that is the tool working, not failing.

What's next

A seeded evaluation set with known ground truth, reporting precision on the Verified tag — turning "we check claims" into a number. Then source tiering, so primary and official sources outrank blogs, and staleness weighting for evidence older than a year.

Built With

• framer-motion
• gpt-oss
• groq
• llm
• next.js
• node.js
• react
• server-sent-events
• tailwindcss
• tavily
• typescript
• vercel
• web-search