Inspiration
You don't need to hack a voting machine to mislead voters. Changing one number on a county results page, or one date on a translated "how to vote" page, can do it. Election offices run small teams and many static pages, often in several languages. We wanted an agent that watches those pages and helps officials decide fast, without ever letting an AI make the final call.
What it does
Nemotron Integrity Agent guards a fictional county election site ("Kanaloa County"):
- Results Seal (no AI): a SHA-256 seal on certified results catches 100% of changes. Every mismatch ends in ROLLBACK (recommended), ESCALATE or HUMAN_SIGNOFF. Nothing is accepted silently.
- Tamper triage (Nemotron Super, run twice): classifies each change as legitimate, tampering or uncertain with evidence. ROLLBACK is recommended only when both runs agree on tampering with confidence ≥ 0.8; any disagreement escalates to a human.
- Incident report (Nemotron Ultra): written only for confirmed rollbacks. It never claims the rollback already happened; an official confirms or reverses it.
- Multilingual consistency: Nemotron Lightning extracts election facts from English, Vietnamese, Spanish and Tagalog pages; a deterministic diff finds drift; Super explains the voter impact and drafts a fix.
- Misinformation hunter (Tavily + Nemotron): live Tavily Search and Extract look for pages contradicting the official facts; Lightning triages, Super writes a report that cites URLs.
- Risk signals for officials: deterministic flags (service-account actor, off-hours change, ticket scope mismatch, one-sided vote removal, registered-voter drop) shown to the human signer. They are informational and not used by the gate.
How we built it
Python with an OpenAI SDK client pointed at Nebius Token Factory. Model routing:
| Task | Model |
|---|---|
| Extraction, triage of search results | Nemotron 3.5 Lightning (chosen by eval) |
| Tamper triage, explaining mismatches | Nemotron 3 Super 120B |
| Incident report (rare) | Nemotron 3 Ultra 550B |
Strict JSON schemas validated locally, bounded retries, token caps, and a usage log of model, tokens, latency and cost per call. Prices are read from Token Factory's model list, and zero data retention is enabled. The demo runs on Render with server-side keys, per-visitor rate limits, a strict CSP, and "Replay" for recorded runs.
Evaluation (honest numbers)
Held-out sets were committed before any run (5c62a31, 81e2b8e, e488cda) and the triage prompt is frozen by hash.
- Translation drift: Lightning 14/14 in both runs (precision 1.00, recall 1.00); Nano 3–5/14, so Lightning became the default.
- Gate on held-out H08–H14 (n=14): v1 8/14 vs v2 9/14 correct; disguised tampering sent to sign-off as "legitimate" 5/6 (v1) vs 3/6 (v2); false rollbacks 1/8 for both.
- Because the Seal detects every change, Nemotron errors affect how urgently a human sees a change, not whether it is detected.
- A full v2 triage costs about $0.003; the whole eval cost $0.13.
Challenges we ran into
Run-to-run variance even at temperature 0, which led to the two-vote gate. Reasoning tokens counting against max_tokens. Disguised tampering with plausible tickets, which the model often believes, so we added risk signals for the human. An ambiguous Vietnamese label that the extractor read correctly and we had written badly.
Accomplishments that we're proud of
An evaluation we can defend: held-out sets committed before running, misses reported instead of hidden, and an AI that is never the sole decider on an election result.
What we learned
Deterministic checks should own detection; LLMs are most valuable explaining and prioritizing. Measuring small and cheap models against each other on your own task beats guessing.
What's next
A larger held-out set, a pilot with a real election office's public pages, screenshot comparison with Nemotron Super VL, and more languages required under the Voting Rights Act.
Log in or sign up for Devpost to join the conversation.