The Warden

A Slack security agent where a human-written policy — not the AI — decides what gets auto-fixed, what needs approval, and what's hard-blocked, powered by live CVE exploitability data and MCP server.

Comment

Inspiration

Every engineering org drowns in the same problem: scanners fire off hundreds of CVEs a week, and almost nobody can tell the difference between "theoretically dangerous" and "actively being exploited right now." Meanwhile, AI agents are being handed more and more autonomy with almost no answer to the one question that actually matters: what is this agent allowed to do without asking a human?

Warden exists at the intersection of those two problems. Instead of treating "how dangerous is this CVE" and "how much freedom should the agent have" as the same question, we split them apart — and let a human-authored policy file, not the AI, own the second one.

What it does

Warden is a Slack agent that scans a repository's dependencies, enriches every finding with live exploitability data (CISA's Known Exploited Vulnerabilities catalog and FIRST.org's EPSS scores), and runs each one through a deterministic OPA/Rego policy gate. That gate — written entirely by a human, never by the AI — assigns every finding to exactly one lane:

  • Block — critical, actively exploited (KEV-listed), or high-EPSS. No approve button exists for these. Policy is absolute; no human can click past it.
  • Ask — a fix exists, but it needs a human's approval before anything happens.
  • Auto — low-risk, a fix exists, and tests are green, so the agent opens (and can merge) the fix on its own.

The result posts to Slack as a color-coded digest. From there:

  • Explain pulls the exact policy reasoning — severity, EPSS, KEV status, the specific rule that fired — into a threaded reply. No guessing, no AI narration, just the gate's own logic formatted for a human.
  • Approve fix calls out to GitHub in real time: creates a branch, bumps the vulnerable pin, and opens a genuine pull request. The Slack card updates itself with a live link.
  • Snooze defers a finding without silently losing track of it.
  • Ask a question in the thread — "why is this blocking?", "how does this compare to the other one?" — and a conversational layer, grounded entirely in the same MCP tools with no ability to open PRs or act, answers using real data. It can explain the gate's decision. It cannot change it.

All of this is exposed as a real MCP server — the same tools Slack calls are independently callable from the MCP inspector or Claude Desktop. Slack is just one client of Warden's brain, not the whole of it.

The core idea underneath all of it: every engineering org drowns in CVE noise, and almost nobody can tell "theoretically dangerous" from "actively being exploited right now." At the same time, AI agents are being handed more and more autonomy with no real answer to the question that matters most — what is this agent allowed to do without asking a human? Warden splits those two questions apart, and lets a human-authored policy, not the AI, own the second one.

How we built it

  • Policy core: OPA/Rego (gate.rego), evaluated via subprocess, fully unit-tested with opa test. This is the one piece of the system that is never touched by an LLM.
  • Enrichment: live calls to the CISA KEV catalog and the FIRST.org EPSS API, with safe fallbacks so a dead feed never silently drops a finding.
  • Agent core (core/): a headless Python engine — no Slack, no MCP, no LLM imports — that turns raw scanner output into policy decisions. Fully testable on its own, which let us prove the entire brain worked before a single Slack message was ever sent.
  • MCP server: a small FastMCP server exposing four tools — scan_repo, get_findings, explain_finding, draft_fix_pr — as thin adapters over the core engine.
  • Slack surface: slack-bolt in Socket Mode, Block Kit for the digest and action buttons, and a conversational layer using Groq's free tier for tool-calling — deliberately scoped to read-only tools, so the part of the system that talks can never act.
  • Live remediation: the GitHub REST API, used to genuinely branch, patch, and open pull requests against a seeded demo repository with real, independently-verified CVEs, including one on the current CISA KEV list.
  • Hosting: Dockerized, with OPA baked directly into the image, deployed on Render, with a scheduled keep-alive so the agent stays reachable through the full judging window rather than just the demo recording.

Challenges we ran into

  • Getting the autonomy boundary right. The hardest design decision wasn't technical — it was making sure a button could never override a block decision. It would have been easy to add an "override" action for convenience. We didn't, because the moment a human can click past the policy, the policy isn't actually a boundary anymore.
  • Keeping the LLM honest. Early on, the conversational layer occasionally answered from the wrong context and asserted things that weren't true. We traced it to a repo-resolution bug — the wrong string was being parsed out of the Slack message — and fixed the root cause rather than papering over it with a better prompt. A wrong answer from an agent that's supposed to be trustworthy is worse than no answer at all.
  • Free-tier everything. Every model, every host, every API in this project runs at zero cost, which meant working around real constraints (rate limits, cold starts, sunset model versions) instead of reaching for a credit card.

Accomplishments that we're proud of

  • A policy engine that is genuinely deterministic and auditable — every decision Warden makes can be traced to an exact rule in a file a human wrote and can read.
  • A real, independently-callable MCP server, not a thin wrapper — the same tools power both the Slack agent and a standalone MCP client.
  • The full loop actually works end to end, live: scan → policy decision → Slack digest → one-click approval → a genuine pull request on GitHub. Nothing in the demo is simulated.
  • A conversational layer that can explain but never act — proving that "helpful" and "in control" aren't the same axis, and you can build for both without trading one off against the other.

What we learned

That the interesting problem in agentic AI right now usually isn't "can the agent do the task" — it's "who gets to decide what the agent is allowed to try in the first place." Policy-as-code, a discipline that already exists in DevSecOps for gating CI/CD pipelines, turned out to map almost perfectly onto that governance question for AI agents. We also relearned a smaller but sharper lesson: when an agent gives a wrong answer, the instinct to fix it with a better prompt is usually wrong — the real bug is almost always in the data reaching the model, not the model itself.

What's next for Warden

Reachability analysis, so exploitability accounts for whether the vulnerable code path is actually called. Support for more dependency ecosystems beyond Python. And extending the same pattern beyond CVEs entirely — the underlying idea, a human-authored policy bounding an AI agent's autonomy, generalizes well past security into any domain where an agent needs real, trustworthy limits.

Note: the current demo uses a seeded fixture dataset for deterministic, repeatable results — swapping in a live scanner feed (Trivy, Snyk, etc.) is the natural next step and doesn't require changes to the policy engine, enrichment layer, or MCP server.

Built With

Share this project:

Updates