sAIfe

sAIfe fact-checks AI answers claim by claim: a second AI judges, code confirms the evidence, failures are fixed or removed, and anything unverified is disclosed.

Comment

Inspiration

Language models sound exactly as sure of themselves when they are wrong. In a chat that is annoying; in an agent it is dangerous, because a made-up fact in step two quietly becomes the input to step three, and by the final answer nobody can tell which sentence was invented.

Two things bothered us most about the usual fixes:

  1. Self-review is often the same model with the same blind spots. Asking a model "are you sure?" changes its tone more than its accuracy.
  2. Safety checks fail silently. When a verifier times out or returns garbage, most pipelines ship the answer anyway, and nothing on screen says the check never ran.

So we set ourselves a Safety-by-Design goal: build a guard whose safety properties live in the architecture, not in a prompt asking the model to be careful. That became sAIfe: every claim checked, every change shown, nothing passed off as true.

What it does

sAIfe takes an AI answer (its own, or one you bring), splits it into atomic claims, and checks each claim with independent detectors:

Detector Catches How evidence is checked
Grounding Claims your sources don't support or contradict A judge returns exact quotes; code confirms each quote is really in the source
Self-consistency Facts the model states differently on each resample Agreement across $N$ samples; can only reach LIKELY, never SUPPORTED
Web evidence World-knowledge errors when no sources are given Search excerpts; quotes checked against the cited text
Deterministic Wrong arithmetic, impossible dates, dead links, invented Python APIs Pure code, no model at all

Flagged claims are corrected (only with a verified quote), hedged or removed, and every edit is recorded. The answer ends as Verified, Caution or Blocked, with plain-language alerts about what was changed and what could not be checked. Users can watch every verdict on a live dashboard they deploy themselves.

How we built it

Eight principles, each enforced by code and by a test. Fail closed; evidence checked by code; independent verifiers; bounded loops; no silent edits; abstention is a valid outcome; untrusted text is data; full audit trail.

Evidence is a string-matching problem, not a trust problem. A judge's quote $q$ counts as evidence for source $s$ only if

$$ \text{verified}(q, s) \iff \operatorname{norm}(q) \sqsubseteq \operatorname{norm}(s) \;\wedge\; |q| \ge 12, $$

where $\operatorname{norm}$ applies Unicode NFKC, straightens quotes and dashes, drops zero-width characters and collapses whitespace, and $\sqsubseteq$ means "is a substring of". A model can never set verified = true; only this function can.

Agreement is not evidence. For consistency we resample $N$ answers and compute

$$ a = \frac{#{\text{samples supporting the claim}}}{N}, $$

but no value of $a$ can make a claim SUPPORTED: the best it earns is LIKELY.

The safety loop can't run away. Each pass verifies only new or changed claims. With $F_k$ the set of flagged claims after pass $k$, the loop stops at the first of: no high-risk claims left, $k = k_{\max}$, the token or time budget spent, or no progress, $|F_{k+1}| \ge |F_k|$. Anything still unchecked becomes UNVERIFIABLE, never SUPPORTED.

Stack. Python 3.13 (uv, Pydantic v2, asyncio), a role-based router across OpenAI-compatible endpoints, OpenRouter and Anthropic, a CLI, a Python SDK and a FastAPI server with live server-sent events. The demo UI and the sAIfe Monitor dashboard are React 19 + TypeScript + Tailwind. The dashboard runs on Vercel's free tier with free Upstash Redis, and receives HMAC-signed webhook events from each user's own sAIfe.

Runs on your own keys. The only required key is a verifier key (a free Google AI Studio key works). A free OpenRouter model can write answers, so a different lab's model always judges them.

Tested like a safety system. 914 offline tests run in under 20 seconds with faked models: property-style invariant suites (no unverified support, failures always UNVERIFIABLE, every edit recorded, loops always terminate, exactly one terminal event) and a red-team suite (tag breakouts, homoglyph quotes, prompt injection, a judge that obeys injected instructions). The dashboard adds 13 API and signature tests. We built it with Claude Code, using parallel sub-agents for independent modules.

Challenges we ran into

  • "Accepted" is not "enforced". Our gateway accepted response_format: json_schema but ignored it, so judges returned malformed JSON and true claims ended up flagged. The same model gave 4/18 valid first-try replies through that gateway and 18/18 once the schema went into the prompt (json_object mode). Our probe now tests each mode several times with a realistic schema and picks the one that actually works.
  • Running out of credit mid-build. When our Anthropic credit ran out, sAIfe failed closed exactly as designed, but it forced a redesign: the default routing now needs only one key, and the original Claude routing is opt-in.
  • Free models are not free of trade-offs. A free judge model gave correct verdicts but no quotes, so under our rules every verdict was downgraded to UNVERIFIABLE. Splitting claims with it took 58 s. Free models now only write answers; judging stays on a model that cites.
  • Web search that returned nothing readable. One search tool version returned no citable text. Switching versions and batching four claims per search cut a strict check from \$2.22 / 186 s to \$0.47 / 41 s.
  • Repair is harder than detection. Live runs surfaced bugs no unit test had: removals leaving sentence fragments, doubled hedges ("reportedly … reportedly"), HTML escapes leaking into repairs, and Verified shown over hedged text. Each fix came with a new test.
  • Same-model agreement can be confidently wrong. In one live run, a model invented math.clamp. Our deterministic detector caught and removed it, but the model's own resamples agreed on the invention, so the leftover text was still marked as LIKELY. It is a vivid reminder of why agreement never counts as evidence, and it is on our list to tighten.

What we learned

  • Make the safe path the only path. Once "evidence" meant "a quote code can find", a whole class of judge hallucinations simply stopped mattering.
  • Measure what you claim. Our benchmark separates planted errors from clean passages and tracks the residual hallucination rate

$$ R = \frac{#{\text{planted errors still in the final answer}}}{#{\text{planted errors}}}, $$

alongside precision, recall, false alarms and silent failures, which must be exactly zero. A preliminary 5-item run took $R$ from 100% to 0% with 9/9 errors detected and 0 silent failures; the full run is next.

  • Independence is a configuration choice. "A second model checks the first" is only true if they come from different labs; we made that explicit in routing and in the docs.
  • Failing closed is a product feature. An honest "I couldn't check this" is more useful than a confident guess, and it is what makes Verified mean something.

What's next

The full benchmark run, an MCP server so any MCP-capable assistant (including Claude Desktop chat) can call verify_answer, and stricter handling of agreement-only claims so the answer status never reads Verified on agreement alone.

Built With

Share this project:

Updates

Submission history