AgentGuard

"The AI that attacks your AI before attackers do

Comment

Inspiration

AI agents are shipping with real permissions now — filesystem access, shell execution, cloud credentials, GitHub, databases. That's what makes them useful, and it's exactly what makes a compromised one dangerous. Most tooling around agent security stops at listing what an MCP config or agent manifest declares. Nobody answers the harder question: if this agent gets compromised — through prompt injection, a malicious tool response, whatever — what could it actually do? We wanted a tool that treats AI agent security the way mature software security already works: don't just review the config, attack it, and prove the fix holds.

What it does

AgentGuard is a pre-deployment security validator for AI agents. You upload an agent or MCP configuration, and it walks through eight stages: it parses the agent's tools and permissions into a threat model, runs an AI red-team simulation that plans and executes a realistic attack chain against that exact configuration, scores the resulting risk, generates a least-privilege policy patch that would stop the attack, and then — the part that matters — replays the same attack against the patched configuration to prove it's actually blocked before calling the agent deployment-ready. It never claims to detect or prevent prompt injection; it assumes the attack succeeds and validates whether the blast radius is survivable.

How we built it

The core of AgentGuard — the FastAPI backend, the four-stage reasoning pipeline, and the Next.js frontend scaffolding — was built in Codex running on GPT-5.6. Inside the app, GPT-5.6 plays four distinct security roles through the OpenAI Responses API, each with its own prompt and reasoning temperature: a security engineer scoring risk (temperature 0.2, for consistent scoring), a red-team engineer planning the attack chain (0.8, for creative exploration within the agent's real permissions only), a security architect designing the minimum-permission fix (0.4), and a verifier re-checking whether the original attack still works after the fix (0.2 again). Every call returns structured JSON, never free text, so the UI can render risk scores, attack graphs, and config diffs directly. After the core build, we used Claude Code for a second pass — cleaning up the generated code and reworking the frontend's component layout and workflow ordering so the UI actually matches an enterprise security dashboard instead of a marketing demo.

Challenges we ran into

The hardest problem wasn't the AI pipeline, it was the UI honestly representing what the pipeline proves. Our first pass buried the actual evidence — the before/after attack comparison and the downloadable fixed config — under a large animated risk-score display, so a validated agent's "proof" screen still looked dominated by the original risk number instead of the fact that the attack was just blocked. We had to explicitly restructure the final screen so the remediated-and-verified result is what a viewer sees first. We also had to resist the instinct to over-animate a security tool — count-up numbers and looping icons read fine on a consumer landing page, but they undercut the "you can trust this assessment" tone a security product needs.

Accomplishments that we're proud of

The attack-replay-then-reverify loop actually works end to end, not as a scripted before/after screenshot but as two real runs of the same AI red-team logic against two different configurations. Getting GPT-5.6 to play four different, temperature-tuned security roles against one shared agent context — and have that translate into an honest before/after (same attack, same tool, different outcome) — is the core proof point of the whole product, and it holds up.

What we learned

The most convincing security claim isn't a score, it's a replay. Watching the same attack fail after the fix communicates "this is deployment-ready" far more credibly than any risk number could — dashboards should be built around demonstrating proof, not displaying metrics. We also learned that reasoning "temperature" is itself a security design decision worth exposing: scoring risk and planning an attack are different cognitive tasks, and tuning each prompt's exploration/consistency trade-off separately made both more reliable than using one setting for everything.

What's next for AgentGuard

  • GitHub Actions / CI integration so agent configs get validated automatically before merge
  • A Cursor / Claude Desktop extension so the same validation runs where agents are actually configured
  • OpenAI Agent SDK integration for runtime — not just pre-deployment — validation
  • Team policy management and compliance reporting for organizations running many agents at once

Built With

Share this project:

Updates