Supervised DevSecOps: AI Agents + Human Gates

A supervised DevSecOps pipeline: GitLab Duo agents remediate security findings and write the tests, while protected branches and manual gates keep humans in charge of every irreversible step.

Comment

Inspiration

Every DevSecOps demo shows a scanner finding a vulnerability. Almost none show what happens next — the triage, the fix, the regression test, the promotion, the deploy. That "life after code" is where engineering time actually goes, and it's exactly what the GitLab Duo Agent Platform is built to automate.

I wanted to answer a real question: can AI agents run the entire post-code lifecycle of a security finding — from scan to production — with a human supervising every irreversible decision? To make the answer honest, I needed a target with real vulnerabilities, so I built VulnMart: a deliberately vulnerable e-commerce app (SQLi, XSS, IDOR, hardcoded secrets, the works) with a runtime scoreboard that proves each exploit works.

What it does

A fully supervised, agent-driven DevSecOps pipeline where each long-lived branch maps 1:1 to an environment on GCP:

  • develop → SAST, secret detection, dependency & container scanning → auto-deploy to dev VM → live DAST scan against the running app
  • staging → unit tests + smoke test → auto-deploy to UAT VM (no re-scanning — findings were already gated on develop)
  • production → build → manual approval gate → deploy to prod VM → read-only health verification, plus a rollback button

Three custom Duo Agent Platform flows close the loop:

  1. Vulnerability remediation — mention the agent on a finding issue and it creates a fix branch, applies the minimal parameterized-query fix, writes a regression test that fails on the vulnerable code, and opens an MR to develop. It only touches Critical/High findings and updates the tracking issue status when done.
  2. Test author — mentioned on a promotion MR, it writes unit and smoke tests for the incoming changes and commits them to the MR's source branch.
  3. Promotion — mentioned on a tracking issue ("promote develop to staging"), it creates a promote/* branch and opens the promotion MR — because service accounts can't push to protected branches, this is the sanctioned path.

The autonomy level is supervised by construction: agents scan, triage, fix, test, and deploy to dev/UAT on their own, but every merge request requires human review, agents cannot push to protected branches or merge their own work, and the production deploy waits for a human to press Play.

How I built it

  • VulnMart: Node.js + Express + EJS + better-sqlite3, with eight deliberate CWE-mapped vulnerabilities and a runtime scoreboard that marks a challenge "detected" when an exploit actually works.
  • Pipeline: GitLab CI/CD with GitLab's managed security templates (SAST, Secret Detection, Dependency Scanning, Container Scanning, DAST) wired to branch-scoped rules so each branch runs only what its environment needs.
  • Infrastructure: three GCP VMs (dev/UAT/prod) running self-hosted GitLab runners tagged per-VM; Docker builds and deploys go through the host socket on protected refs only; MR builds are isolated on the hosted pool.
  • Agents: three custom flows defined in YAML (deterministic fetch step → agent step), grounded by AGENTS.md project instructions, MR review instructions aimed at our vulnerability classes, and a remediation cheatsheet that turns findings into one-look fixes.

Challenges I ran into

  • The scanner missed our own vulnerability. I added a fresh SQL injection for the demo — and Semgrep stayed silent. Root cause: GitLab's managed ruleset has no sink for better-sqlite3's .prepare(); it only models mysql/pg-style sinks. I extended the ruleset with a project-level sast-ruleset.toml (keeping all managed rules) plus a custom taint rule, and verified it end-to-end in CI.
  • The analyzer's YAML parser is stricter than Semgrep's. A well-meaning review suggestion grafted a taint block onto our rule and left a duplicate metadata: key and a backtick-quoted scalar — plain Semgrep accepted it, GitLab's analyzer hard-failed with exit status 7. I now reproduce parser behavior locally with Go's yaml.v2 before pushing rule changes.
  • Security reports uploaded but never ingested. The Vulnerability Report stayed empty despite valid artifacts. The first cause was ours — the default branch was production, but scans run on develop, and GitLab only ingests from the default branch. Even after fixing that, the trial namespace's report processing returned nothing, so I designed the remediation flow to be driven from issues rather than depending on the report.
  • Agents can't push to protected branches. Flows act with the service account's Developer role, so a direct develop→staging MR is impossible for them. The promote/* branch pattern emerged from that constraint — and turned out to be the cleanest possible audit trail for promotions.
  • Our own agents caused a merge conflict. The test author committed redirect tests on staging while the remediation fix landed order-tracking tests on develop — same anchor point, both branches. Resolving it meant keeping both test blocks; a very human problem created entirely by agents.

Accomplishments that I am proud of

  • The full loop works with zero human code written after the finding: scan → issue → agent fix + regression test → MR → human review → auto-deploy → DAST re-scan → promotion MR with agent-written tests → human merge → production behind a manual gate.
  • I extended GitLab's SAST with a custom sink and watched it catch two Critical SQL injections the managed ruleset had never flagged — including one that had been sitting in the codebase all along.
  • Eight pipeline stages touched (plan, create, verify, package, secure, release, monitor), all on GCP infrastructure provisioned for this submission.
  • The safety properties aren't policy documents — they're structural: protected branches, service-account role limits, and a manual production gate make the "supervised" level unenforceable-proof.

What I learned

  • Agent autonomy is a permission architecture, not a prompt instruction. The strongest guardrail isn't telling the agent "never push to develop" — it's that the platform makes it impossible.
  • Scanners are only as good as their sinks; knowing when to extend the ruleset is a core DevSecOps skill, and validating custom rules against the analyzer's actual parser saves a broken pipeline.
  • Designing agent workflows around constraints (protected branches, role scopes) produced better patterns than I would have chosen unconstrained — the promote-branch flow is the clearest example.
  • End-to-end verification beats per-stage success: every stage of our pipeline was "green" while the Vulnerability Report silently stayed empty.

What's next for Supervised Pipeline - AI + Human In The Loop

  • Auto-triage trigger with a severity gate: fire the remediation flow on work-item creation, but only for Critical/High findings — the intake gate already exists, the trigger turns "mention to start" into "file to start".
  • Duo Vulnerability Resolution integration once the Vulnerability Report ingestion is sorted on our namespace, so scanner findings become remediation issues automatically.
  • Slack/Teams notifications at the human gates (MR ready, production awaiting approval) to cut the polling loop.
  • Policy-as-code enforcement of the supervised model via compliance pipelines, so the human gates are provably non-bypassable.
  • Generalizing the flow pack (remediation, test author, promote) into a reusable template any GitLab project can adopt.

Built With

Share this project:

Updates

Submission history