Inspiration
What it does
How we built it
Challenges we ran into
Accomplishments that we're proud of## Inspiration
Ask any AI assistant whether your plan is good and it will find reasons it is good. Frame a question and it accepts your framing. That makes the entire category useless for the one moment you actually needed help: deciding something.
The failure isn't a knowledge gap, it's a structural bias toward agreement — and you can't prompt your way out of it, because the thing you'd be asking is the thing that agrees with you. So I built the opposite: an engine that is structurally incapable of agreeing with you, because no single agent in it is allowed to hold your position.
What it does
You submit a claim. Two agents are assigned opposite sides, and neither chooses which side it defends.
- Opening — both written blind, so neither can react to the other.
- Rebuttal — each side must take apart what the other actually said, quoting the exact phrase. Introducing a new argument of your own is forbidden.
- Cross examination — one question each, and the answer has to lead with the answer.
- Open clash, 0–6 rounds — only while a referee rules the disagreement still live. The referee names the exact tension left open, and that sentence becomes the next round's instruction.
- From the bench — the judge may put one question to a side, or to both, and is told to ask only when something load-bearing has gone undefended.
- Last word, then the verdict: both sides scored 0–100, a winner, a confidence, the strongest and weakest arguments quoted, and the question neither side settled.
- Challenge — you argue back. Both sides must answer you, and the debate is judged again. The judge is told your argument carries no authority from you having started the debate — and that it may still legitimately move the verdict.
There is no language setting. It argues in the language of your claim, verdict included.
How we built it
FastAPI streams the debate over server-sent events; React types it out; the whole thing deploys as one container serving one URL.
The AI is one model behind four different system prompts:
- Two debaters stream token by token. The prompts carry the product logic: distinct temperaments so the two don't sound alike, a ban on the essay-prose connectives that make transcripts read as machine output, a rule that every turn after the opening must quote and attack the opponent's actual words, and a scoring reward for conceding a point you can't answer.
- A referee decides after every round whether the argument still has anywhere to go, as a structured output — so the debate's length is model-decided within bounds, not a constant in the code.
- A judge asks its own question before closing statements, then returns the verdict as a validated object rather than prose that has to be parsed hopefully.
Two provider adapters (Anthropic, Mistral) sit behind one interface, and the debate protocol never knows which it's talking to.
Challenges we ran into
The two agents sounded identical. Two monologues in the same register is indistinguishable from two opinion generators sharing a page. Fixed by giving them opposed temperaments, forcing second-person address, requiring a short exact quote before every attack, and banning "Furthermore / My opponent argues that / While it is true that" outright — plus banning the single biggest tell: opening a turn by summarising what the other side said.
They invented citations. A fabricated "2022 meta-analysis" is the worst failure mode here, because a reader can't check it mid-debate. So named sources are banned absolutely — argue from mechanism instead — and the judge is instructed to punish any citation heavily, whether or not it happens to be true.
Markdown leaked into speech. Asterisks showed up as literal punctuation mid-sentence. A prompt rule gets obeyed most of the time; "most of the time" isn't good enough, so the character is now stripped deterministically on the way out.
Fixed round counts made every debate feel scripted. Four rounds ends debates that are still going somewhere and pads ones that are finished. The referee replaced the constant.
Streaming looked like paste, not argument. The model writes a paragraph faster than anyone reads one, so raw SSE arrived in bursts. The client now queues every event and releases characters on a clock, pausing at sentence ends and speeding up only when it has fallen several turns behind — which also keeps the order honest, since a verdict can't appear while the last word is still being typed.
Free-tier quotas died mid-debate. A twenty-call debate exhausts a free tier, and backoff meant thirty seconds of waiting for the same refusal. API keys are now a rotating ring: an exhausted key moves to the next and retries immediately.
A demo that depends on a live model is a demo that fails on stage. So a debate can be recorded and replayed down the same event stream — no key, no network, no second code path — and the app labels a replay as a replay. That's also what makes the static deployment work.
Accomplishments that we're proud of
- The judge is scoring argumentation, not truth: defending a false claim skilfully beats defending a true one badly. It's also told the claim having been submitted is not evidence for it, and that attacking isn't inherently stronger than defending — the two failure modes that would quietly turn the verdict back into flattery.
- The challenge round is the obvious place for the whole thing to collapse into agreement, and it's the most constrained turn in the protocol instead.
- The debate's length is decided by the argument, and the referee's reasoning is shown on screen so the reader can see why it kept going.
- 30 tests that need no network and no API key, covering the parts that actually break: round length, the cap, the optional bench question, asterisk stripping, verdict reconciliation, referee failure, key rotation, and replay with no engine configured at all.
- It works in any language with no language setting, verified end to end in English and Turkish.
What we learned
Prompts are where the product logic lives in a system like this — the difference between a debate and two monologues is a dozen sentences of instruction, not a bigger model. Structured outputs are what make a model's decision usable as control flow, which is how the referee ended up deciding round count. And a model constrained honestly (concede what you can't answer) argues better than one told to win.
Also: never trust a prompt rule for something you can enforce in code.
What's next for Devil's Advocate
- Shareable debate permalinks, so a verdict can be sent to the person who disagreed with you.
- A calibration harness for the judge: replay the same transcript across models and measure whether the scores agree.
- "Argue against this" as an integration — a plan, a PR description, a strategy memo — where the claim is pulled from the document.
- Local models, so a debate about something confidential never leaves the machine.
Log in or sign up for Devpost to join the conversation.