Falsify

Verifying AI truth with incentives and rules — not another AI.

Comment

Inspiration

AI now generates answers, code, and data faster than anyone can manually verify — and when an AI answer is wrong, the cost lands on the person who used it, not the model. The obvious fix, "ask another AI," doesn't work: a second model's opinion is itself unverified, so you've just traded one unverified claim for two. Falsify flips this. Instead of trusting a model, put a bounty on the claim and pay people to break it.

What it does

Falsify is a bounty market for truth. A customer stakes USDC behind a claim and a committed test. Attackers submit counterexamples. A deterministic verifier — not another AI — runs the test: if the counterexample breaks it, the finder takes 85% of the bounty and Falsify keeps 15%. If nothing breaks it, the claim survived a paid attack, which is proof a green test suite can't give.

How we built it

  • Gemini generates counterexamples for correctness claims as structured JSON.
  • For security claims, Gemini won't write exploits — its safety filters block it — so a deterministic attack harness produces the attack and Gemini only explains it.
  • Foundry's forge test is the deterministic verifier; the exit code is the verdict, and no LLM decides the payout.
  • A Solidity settlement contract escrows USDC, gates submissions by ERC-8004 identity, writes reputation, and settles.
  • Cloud Run + Firestore (persistent evidence) + Secret Manager, with a MetaMask wallet flow that escrows USDC directly from the browser.
  • The full loop ran on Base Sepolia with real Circle USDC — 5 transactions verifiable on Basescan.

Challenges we ran into

  1. Gemini refuses to generate exploit code. We turned the blocker into the design: security counterexamples come from a deterministic harness, correctness counterexamples from Gemini.
  2. Every shortcut back to "AI verifies AI" reintroduces an unverified opinion. The deterministic test is the only trust anchor.
  3. Real on-chain settlement — funding a wallet, minting USDC, wiring browser-wallet transactions — was the long tail that turns a demo into verifiable evidence.

Accomplishments that we're proud of

  • A working product, live on Cloud Run, that calls Gemini and settles deterministically.
  • A real USDC settlement on Base Sepolia, verifiable on a block explorer.
  • The honesty of the design: Gemini proposes counterexamples, but a deterministic test always decides — "agents propose, rules dispose."

What we learned

The mechanism matters more than the model. Three Gemini personas independently found the same counterexample, but the verdict came from the test, not from any of them. Incentives and deterministic rules — not another AI's opinion — are what make verification trustworthy.

What's next for Falsify

  • Persistent on-chain identity and reputation, plus a real mainnet USDC settlement.
  • Continuous adversarial testing: auto-bounty every new claim on every deploy.
  • A marketplace of verifiers and adversarial agents, with the platform take rate as the revenue engine.

Built With

Share this project:

Updates