Agentshield

Secure AI agents before they act

Comment

Inspiration

AI agents don't just answer questions anymore. They call tools, read and write data, remember things across sessions, and increasingly talk to other agents. That's the whole point of building one — but it's also the problem. The more autonomy you give a model, the more ways there are for it to go wrong.

We kept running into the same gap. Traditional security scanners were built for a world where the worst thing untrusted input could do was corrupt a SQL query or slip in some XSS. They don't have a category for "a prompt that talks the model into ignoring its own instructions," or "a memory entry that poisons every future session," or "a tool description that smuggles in a command." None of that looks like a syntax problem, so a syntax-level scanner walks right past it.

What teams actually do about this right now is stitch together a static scanner, a manual review, and maybe a red-team pass, and then try to reconcile three completely different outputs by hand — usually after the agent is already live. We wanted one workflow you'd actually run before that, not after.

What it does

Point AgentShield at a repo and it runs a layered pipeline:

  1. Rules-engine scan — Semgrep rules catch known-bad Python/Java patterns: hardcoded credentials, unsafe eval/tool-call patterns, unbounded LLM calls, unvalidated peer-agent input, that kind of thing.
  2. Manifest scan — a scanner we wrote for agent-skill manifests (SKILL.md), checking permissions, metadata, embedded code, and dangerous combinations across skills.
  3. Codex security review — Codex reads the whole repo against a 62-control contract, decides whether every static finding is a true positive, false positive, or "depends on context," and flags new issues the static rules never could.
  4. Codex Behaviour Emulator — the part we're most proud of. Four Codex roles (Planner, Attacker, Agent, Judge) work together to map entry points, match untrusted sources to real attack techniques — prompt injection, memory poisoning, tool description poisoning, authority spoofing — and predict how an attack would actually move through the code. It attempts up to 8 attacks per attack class before giving up or stopping early if one lands, and every claim comes with file:line evidence. All of this happens from source code alone, no live endpoint needed.
  5. Optional live probe — if you want runtime proof instead of a prediction, it can replay adversarial payloads against a real endpoint.
  6. Unified report — one HTML/Markdown/JSON/SARIF report with ranked findings, attack paths, OWASP/MITRE ATLAS/CWE mappings, and actual fixes, instead of five disconnected outputs you have to reconcile yourself.

Basically, it doesn't just check that a security control exists. It checks whether it actually works.

How we built it

The deterministic side is a Semgrep rule pack plus a manifest scanner we wrote from scratch, covering Python and Java. On top of that we built a fairly strict integration with Codex — we don't just ask it an open question and hope for something useful back. Every stage gets a repo-specific prompt and a strict JSON schema, and if Codex's output doesn't validate against that schema, the merger won't touch it. It runs entirely on the user's own ChatGPT-authenticated Codex CLI (agentshield/codex_runner.py), so there's no separate API key to manage.

The Behaviour Emulator took the most iteration. Early versions were a single prompt trying to plan an attack, execute it, and grade itself, and honestly it showed — the output was plausible but not trustworthy. So we split it into four roles instead: Planner picks the entry point and untrusted source, Attacker generates the challenge and adapts if something blocks it, Agent traces the execution path and records evidence, and a separate Judge reviews everything the first three produced and throws out anything that isn't actually supported. That last part matters more than it sounds like it should — having a role that can only delete claims, never add them, is what turned this from "an LLM's guess" into something we'd actually put in a report.

Challenges we ran into

Getting to produce output we could trust was the biggest one. The first pass at the emulator would sometimes report an attack "landing" when it hadn't really traced through the code carefully enough to back that up. Adding the independent Judge role — one whose only job is to strip unsupported or duplicate claims — fixed most of it, but it took a few rounds to get there.

We also had to decide how much authority to give Codex over the static findings. It would've been easier to just let Codex override or delete anything it disagreed with, but that throws away the repeatability you get from deterministic rules. So instead Codex annotates static findings (true positive / false positive / context-dependent) rather than deleting them — more work to build, but it means nothing silently disappears.

The other recurring issue was language. A prediction that an attack could work is not the same thing as proof that it does, and it was surprisingly easy to accidentally blur that line in the report UI. We ended up treating "predicted" (from the offline emulator) and "proven" (from the optional live probe) as two different claims everywhere, down to using different colors for each.

Accomplishments that we're proud of

  • One command sequence — scan, codex, merge, check — takes a repo from nothing to a ranked, evidence-backed report.
  • The four-role Codex architecture. It's not flashy, but splitting Planner/Attacker/Agent/Judge apart instead of asking one prompt to do everything is the reason we trust the emulator's output at all.
  • Codex checking Codex. Having the Judge role catch the Attacker/Agent roles' overclaiming before a human ever sees it is probably the single design decision we're most proud of.

What we learned

Securing an AI agent is a genuinely different problem from securing a normal application — the attack surface is the model's reasoning, not just its code paths. We also learned that getting an LLM's output trustworthy enough to put in a security report has almost nothing to do with prompt cleverness and almost everything to do with structure: strict schemas, a role that can only remove claims, and being ruthless about never letting "predicted" and "proven" blur together.

What's next for AgentShield

  • Incremental scans based on Git diffs instead of full-repo rescans every time.
  • Baseline comparison and policy gates so teams can block a release on regressions.
  • Language coverage beyond Python/Java — TypeScript and Go are next.
  • Signed report attestations and exportable evidence bundles.
  • Public benchmark repos with real precision/recall numbers, instead of just our own demo agent's results.

Built With

Share this project:

Updates