Inspiration
My background is DevOps, DevSecOps and platform engineering: building the paved roads teams ship on, from CI templates and security gates to deploy tooling and on-call runbooks. AI can now write the code, but someone still babysits everything after it: review, pipelines, scans, the deploy, and the first hours in production. And no platform team will hand production to an agent without guardrails they trust.
Closed Loop Delivery hands that whole post-code loop to GitLab Duo agents, with the guardrails a platform team would insist on.
What it does
You create an issue and assign it to the Duo Developer agent. That is the only human action. From there:
- Create: the Duo Developer flow implements the change with tests and opens a merge request.
- Review: GitLab Duo Code Review reviews it automatically.
- Verify and secure: the MR pipeline runs unit tests, SAST, secret detection, dependency audit and container scanning.
- Govern: an OPA release gate blocks High or Critical code findings, Critical or fixable High container CVEs, leaked secrets, vulnerable dependencies, missing evidence, unpinned images and out-of-policy scaling.
- Risk-based autonomy: a deterministic merge-risk policy decides whether the MR may merge hands-off. App-code changes do. Changes to the pipeline, policies, infrastructure, dependencies or very large diffs are labelled
needs-human-approvaland held. The agent that wrote a change never decides how risky it is. - Release: on main, the image ships to Google Cloud Run as a zero-traffic canary, is probed side by side against the live baseline, and is promoted or automatically rolled back.
- Report: a GitLab Release records the evidence, and a summary is posted back on the original issue.
- Monitor, back to Plan: a scheduled production health check keeps watching. If it fails, a pipeline-failed trigger starts a custom three-agent Duo flow (investigate, file incident, remediate). The flow diagnoses the cause from job logs and commits, opens an incident with severity and root cause, and proposes a fix MR when the cause is a clear regression.
In the demo, the issue "Add a live status page" went from creation to a live page on Cloud Run in about 15 minutes, with no human action after the issue was created. The agent wrote the page and test, Duo reviewed it, the risk policy classified it as low risk, it auto-merged, passed canary analysis, was promoted, and the issue was closed with a release summary.
How we built it
- GitLab Duo Agent Platform: a custom flow (
.gitlab/duo/flows/incident-triage.yaml) with three agents and least-privilege toolsets. The agent that reads untrusted logs is separated from the agents that act. A pipeline-event trigger starts it. The foundational Developer and Code Review flows handle issue to MR and review. - GitLab CI/CD: MR pipelines with all scanners, OPA/conftest policy-as-code, a JSON merge-risk policy, canary analysis (Python, standard library), auto-merge and issue reporting through the GitLab API, release evidence, and pipeline schedules with typed inputs.
- Google Cloud: Cloud Run (scale to zero, tagged canary revisions, traffic splitting), Artifact Registry, and Workload Identity Federation, so GitLab OIDC tokens deploy with no stored keys. A dedicated runtime service account with no roles.
- Container: a minimal Alpine image running as a non-root user, pinned by digest. Moving off a Debian base cut container findings from 164 to 0.
DevSecOps stages covered (all 9)
- Plan: issue intent; incidents created from production failures
- Create: Duo Developer flow; fix MRs from the triage flow
- Verify: unit tests, automatic Duo Code Review, MR pipelines
- Package: Alpine image pinned by digest, Artifact Registry
- Secure: SAST, secret detection, dependency audit, container scanning
- Release: canary, baseline comparison, promote or rollback, GitLab Releases, issue summary
- Configure: keyless WIF access, least-privilege runtime, scaling enforced by policy
- Monitor: scheduled SLO health check feeding a Duo trigger
- Govern: OPA release gate, risk-based merge policy, merge checks, recorded merge decisions and release evidence
Challenges we ran into
- Triggers do not fire on bot or flow actions, so the monitoring loop starts from a human-owned pipeline schedule. That is cheap CI, and AI only runs when something is actually wrong.
- Noisy canaries: early canaries were rolled back on network noise. Fixing that meant interleaved baseline probing, reused connections and median-based regression checks. The false rollbacks were themselves caught, recorded and turned into incidents by the system.
- Races with the agent: the MR pipeline sometimes started before the agent had finished, and the agent's final update reset the MR to Draft. Auto-merge now waits for the agent to settle.
- A gap in our own gate: the container scan found 164 OS vulnerabilities the gate was not checking. We wired the scan into the policy and moved to a minimal base image.
- Cloud Run reserves
/healthz, which only surfaced against the real service.
Accomplishments that we're proud of
A real, repeatable issue-to-production run with no human in the middle, on live Google Cloud infrastructure, where every decision (review, gate, risk class, canary verdict) is recorded and explainable.
What we learned
Give agents the work that needs judgement, and keep trust in deterministic, versioned policy. That split is what makes hands-off safe enough to adopt.
What's next
Feed production signals to the triage agent through an MCP server on Cloud Run, add per-service SLO budgets, and package the flows and policies as a reusable CI/CD Catalog component so any project can adopt the loop.
