AI Release Guardian

AI Release Guardian analyzes merge requests, flags risks, and guides developers toward safer releases using event-driven GitLab Duo multi-agent workflows.

Comment

AI Release Guardian

Multi-agent MR review system powered by GitLab Duo Agent Platform

Inspiration

In modern software development, writing code is no longer the main challenge — deciding whether that code is safe and ready to release is. Developers spend significant time manually reviewing merge requests, checking for security issues, validating test coverage, and ensuring compliance across fragmented tools and logs.

GitLab provides powerful built-in capabilities, but the review process remains manual and disconnected. I built AI Release Guardian to unify that decision-making into a single, automated workflow — reducing effort and bringing clarity to release readiness.

What It Does

AI Release Guardian is an event-driven GitLab Duo flow that automatically analyzes merge requests and surfaces risks before code ships.

When a merge request is created or updated, the system:

  • Reviews code changes across 10 specialized dimensions simultaneously
  • Identifies security risks — exposed secrets, vulnerable dependencies, dangerous configurations
  • Highlights missing tests — coverage gaps, failing or flaky tests, CI inefficiencies
  • Evaluates compliance — missing approvals, labels, changelog entries, policy violations
  • Assesses architecture — risky coupling, layering violations, oversized diffs
  • Checks performance — runtime hotspots, heavy queries, infrastructure concerns
  • Reviews observability — logging, metrics, tracing, and alert coverage
  • Validates documentation — changelog, user docs, runbooks, config docs
  • Evaluates UX impact — usability, accessibility, error states
  • Analyzes dependencies — supply-chain risk, unpinned packages, lockfile issues
  • Audits CI sustainability — repeated retries, long jobs, noisy failures
  • Produces a structured summary with risk levels and actionable recommendations posted directly on the MR

This eliminates the need to manually inspect multiple tools, dashboards, and logs.

How I Built It

I built this solution entirely on the GitLab Duo Agent Platform using custom flows.

Architecture

                    ┌─────────────┐
         Trigger    │ Build MR    │
     ───────────────│  Context    │
                    └──────┬──────┘
                           │
         ┌─────────────────┼─────────────────┐
         │        │        │        │        │
    ┌────▼──┐ ┌───▼───┐ ┌──▼──┐ ┌───▼──┐ ┌──▼───┐
    │Security│ │ Tests │ │Comp-│ │Archi-│ │Perf- │  ... (+5 more)
    │Review  │ │Review │ │liance│ │tecture│ │ormance│
    └────┬──┘ └───┬───┘ └──┬──┘ └───┬──┘ └──┬───┘
         │        │        │        │        │
         └─────────────────┼─────────────────┘
                           │
                    ┌──────▼──────┐
                    │  Release    │
                    │  Summary    │
                    └──────┬──────┘
                           │
                    Posts MR Note

Key components:

  • 12 custom agents defined in flow.yml, each with a specialized prompt, scoped toolset, and structured output format
  • Fan-out/fan-in routing — one context-building agent feeds 10 parallel reviewers, which converge into a final summary agent
  • Event-driven triggers — the flow activates on MR mention, assignment, or reviewer assignment via the service account
  • CI/CD pipeline with 4 stages: YAML linting, SAST, Secret Detection, and documentation validation
  • Project scaffolding — AGENTS.md for Duo behavior customization, CODEOWNERS for automated review assignment, Kubernetes deployment manifests, wiki documentation, and observability configs

Tech Stack

Component Technology
Platform GitLab Duo Agent Platform
Flow Engine GitLab Custom Flows (YAML)
Agent Type AgentComponent (ambient environment)
CI/CD GitLab CI with SAST, Secret Detection
Infrastructure Kubernetes deployment manifests
Monitoring GitLab Analytics Dashboards

Challenges I Faced

Flow configuration complexity — Correctly structuring the agent and flow YAML with proper input bindings, router definitions, and toolset scoping required careful iteration. A single misconfigured from: reference breaks the entire pipeline silently.

Service account authentication — The flow's service account encounters an insufficient_scope error when calling GitLab API tools. This turned out to be a known GitLab bug (#594248) affecting custom flows on GitLab.com, not a configuration issue on my end.

Prompt engineering at scale — Designing 12 distinct system prompts that produce consistently structured, actionable outputs without overlap or redundancy was a significant design challenge. Each agent needed to stay focused on its domain while avoiding duplicate findings.

CI validation across branches — Ensuring the pipeline works correctly across all branch types (default, feature, patch) required careful rules: configuration — a docs-check job that works on feature branches breaks on branches created from main.

Accomplishments I'm Proud Of

  • 12 specialized agents working in parallel, each with a focused role and scoped toolset
  • Complete project infrastructure — 17+ files across CI/CD, Kubernetes, wiki, observability, and documentation
  • Zero-config triggering — just mention the service account on any MR and the full review runs automatically
  • Production-grade CI pipeline with YAML linting, SAST, Secret Detection, and documentation validation
  • Clean fan-out/fan-in architecture that scales naturally as new review dimensions are added

What I Learned

Event-driven AI is more effective than chat-based AI. When AI operates within the developer workflow — triggered by real events, producing results where developers already work — it becomes genuinely useful rather than an extra step.

Clarity beats volume. Developers benefit far more from concise, risk-rated insights than from large dumps of raw data. A single structured note with "medium risk — 3 findings, 2 actions" is more actionable than pages of verbose analysis.

Multi-agent orchestration is the future. A single monolithic prompt cannot cover security, testing, architecture, performance, and compliance with equal depth. Specialization through multiple focused agents produces significantly better results.

GitLab Duo Agent Platform is powerful but evolving. The platform enables sophisticated workflows, but authentication and scope management for service accounts are still maturing. Early adopters should expect to work around known issues.

What's Next for AI Release Guardian

  • Automated fix suggestions — Agents will propose code fixes, not just flag issues
  • Risk trend analysis — Track risk levels across MRs over time to identify systemic patterns
  • Deeper CI/CD integration — Trigger reviews on pipeline events (failures, security findings) not just MR activity
  • Real-time alerts — Slack/email notifications for high-risk findings
  • Custom rule engine — Let teams define their own review policies via AGENTS.md
  • Cross-MR dependency analysis — Detect when multiple open MRs create conflicting changes
  • Learning from feedback — Use developer responses to improve finding accuracy over time

Built With

  • anthropic-claude
  • automation
  • ci/cd
  • code-analysis
  • flow.yml)-for-defining-agents-and-orchestrating-flows-gitlab-(merge-requests
  • gitlab
  • gitlab-apis
  • gitlab-duo
  • gitlab-duo-agent-platform
  • merge-requests
  • mobile-application
  • responsive
  • risk-detection
  • triggers
  • web-application
  • yaml
Share this project:

Updates