Gauntlet

• 

  the Range grid, unhardened

• 

  the Range under Policy v2

• 

  the fix

• 

  the attack trace with the leaked canary

• 

  v1 and v2

• 

• 

  The Architecture diagram

• 

  Metric Interval Estimates

Inspiration

AI agents now read email, web pages and documents, and any of that text can be written by an attacker. Most "agent safety" demos show one scary jailbreak and stop. We wanted something you can measure: attack an agent at scale, decide breaches with code rather than opinion, fix them, and re-run the same attacks to see whether the fix actually worked.

What it does

Gauntlet is a test instrument for agent safety. It runs a swarm of prompt-injection attacks against an inbox-assistant agent, and a deterministic oracle (not another model) decides what counts as a breach: exfiltration of a secret (M1), an unauthorized action (M2), or unauthorized egress (M3). Failures are clustered, and a Nemotron model writes a policy-as-code fix (Policy v2). Then the same attacks are re-run, together with a benign-task suite that checks the agent still does its job. Every result carries Wilson 95% confidence intervals, and a "real difference" is only claimed when the intervals do not overlap.

Recorded results (fictional company, one agent, synthetic secrets):

All 49 unsafe actions came through the email channel (49 of 150 email attacks); web-page and document attacks produced none. Of the 49, Policy v2 stopped 40 with a named rule; 9 did not recur, because the agent's behaviour varies between runs. On a 45-probe set where the attacker was given the secret, unsafe actions went from 1 of 45 to 0 of 45, and canary-only leaks from 10 of 45 to 0 of 45.

You can also run a small live run (8 attacks, recorded locally in our demo video): 4 of 8 were unsafe unhardened and 0 of 8 under Policy v2. Eight attacks is a demonstration, not a statistic, so the public site shows the recorded run.

How we built it

• Nebius Token Factory (OpenAI-compatible API) serves every model call.
• NVIDIA Nemotron models do the work: Nemotron Super is the agent under test, Nemotron Ultra handles reconnaissance, clustering, defence and the report, and lighter Nemotron models generate attack payloads (with a fallback model).
• Tavily provides runtime search during reconnaissance.
• A FastAPI backend runs the orchestrator, oracle and policy enforcement. A Next.js frontend shows the Range (one square per attack), attack traces, results and a printable verification report.
• A claims audit script checks every number we publish against the recorded data, so the README, site and report cannot drift apart.
• The public demo is a static export of the recorded run deployed on Vercel. Live runs are limited by per-run and daily spend caps and are disabled on the public site.

We did not use Nebius Serverless Jobs or Endpoints.

Challenges

The hardest part was honesty, not code. Early versions of our numbers used different denominators in different places, and our own audit caught them. We rebuilt the pipeline so every figure has a stated denominator and interval, labelled superseded numbers as historical, and made the audit fail on any number it cannot trace.

What we learned

A deterministic oracle changes what you can claim. Policy-as-code is reviewable and reproducible, and the benign-task suite and deny-everything control stop "fixes" that simply break the agent.

Limitations

• One fictional company and one agent.
• The secrets registry matches literal values.
• A labelling effect on benign tasks is not excluded.
• The measurable improvement is in unsafe actions that bypass the agent's rules (M2, M3), not in exfiltration alone.
• Breaches in our data came only through email.

What's next

More agents and tool sets, more channels, and policy generation that works across different companies.

Built With

• fastapi
• nebius
• next.js
• nvdia
• openai-sdk
• python
• react
• sqlite
• tavily
• typescript
• vercel
• yaml