ComplianceBot Flow: From Manual to Instant Complience

A multi-agent AI flow that scans MRs, maps findings to compliance controls, and generates audit-ready evidence - without leaving GitLab Agent Duo Platform

Comment

Inspiration

Compliance audits are broken. Every engineering team knows the drill — weeks before a SOC 2 or ISO 27001 audit, developers are pulled off real work to manually comb through code, fill spreadsheets, and gather evidence. The average SOC 2 audit consumes 200–400 engineering hours, and most of that time is spent on work that should be automated.

The question that inspired ComplianceBot was simple:

"What if every merge request was automatically audited the moment it was created?"

With the GitLab Duo Agent Platform opening up true multi-agent orchestration inside GitLab, we saw the perfect opportunity to build a compliance auditor that lives exactly where the code lives — inside the MR workflow itself.

What it does

ComplianceBot Flow is an AI-powered multi-agent compliance system built on the GitLab Duo Agent Platform. On every merge request, it:

  • 🔍 Scans diffs for security violations — secrets, weak encryption, SQL injection, CVEs, misconfigurations
  • 🗺️ Maps every finding to real compliance controls across SOC 2, ISO 27001, PCI-DSS, HIPAA, and custom org policies
  • 📦 Collects audit evidence automatically, ready for auditors
  • 📋 Reports directly on the MR — posting a compliance score, creating GitLab issues per finding, and generating audit packages

The compliance score is calculated as:

$$ \text{Compliance Score} = 100 - \sum_{i} w_i \cdot f_i $$

Where $w_i$ is the severity weight of finding category $i$ (Critical = 25, High = 10, Medium = 5) and $f_i$ is the number of findings in that category.

A test MR with intentional violations scored 0/100, with 4 Critical, 10 High, and 10 Medium findings mapped across SOC2-CC6.1, ISO27001-A.10.1.1, PCI-DSS-3.5.3, and more.

How we built it

The system is built as a 4-agent pipeline on the GitLab Duo Agent Platform:

Scanner Agent → Mapper Agent → Evidence Collector → Reporter Agent

Each agent is defined as a YAML file and a Python implementation:

  1. Scanner Agent — reads MR diffs via GitLab API, pattern-matches against a library of security rules organized by category and severity
  2. Mapper Agent — takes raw findings and maps them to control IDs across JSON-defined compliance frameworks (SOC 2, ISO 27001, PCI-DSS, HIPAA)
  3. Evidence Collector — aggregates findings, timestamps, MR metadata, and author context into structured audit packages
  4. Reporter Agent — posts a formatted MR comment with the compliance score, creates individual GitLab issues per finding, and optionally archives to GCP

For optional GCP integration, we connected:

  • Vertex AI for AI-generated compliance narratives
  • BigQuery for historical analytics and trend tracking
  • Cloud Storage for PDF report archival with 1-year retention

The entire infrastructure is reproducible with a single ./scripts/gcp_setup.sh command using Terraform.

Challenges we ran into

🔐 GitLab Pipeline Access The biggest blocker was not having Maintainer-level pipeline access during the hackathon. We solved this by building a full local runner (python -m src.local_runner) that replicates the entire agent pipeline offline using demo data — so the system could be demonstrated and tested without CI/CD access.

🤖 Multi-Agent Coordination Designing clean handoffs between 4 agents without state bleed was harder than expected. Each agent needed to be independently testable while also functioning as part of an orchestrated flow. We settled on structured JSON as the inter-agent contract.

☁️ GCP Graceful Degradation We wanted GCP to be optional, not a hard dependency. Building a degradation layer where the flow still produces full compliance reports even when BigQuery, Cloud Storage, or Vertex AI are unavailable required careful error handling at every integration point.

📐 Control Mapping Accuracy Mapping a raw code finding like "MD5 hashing detected" to the right control (ISO27001-A.10.1.1 vs PCI-DSS-3.5.3 vs both) required building a detailed, hand-curated control definition library — a surprisingly deep compliance research task.

Accomplishments that we're proud of

  • ✅ End-to-end multi-agent flow working on a real GitLab MR
  • ✅ 20+ issues auto-created with correct severity, control mappings, and remediation guidance on a single MR scan
  • ✅ 5 compliance frameworks covered out of the box (SOC 2, ISO 27001, PCI-DSS, HIPAA, Custom Org Policies)
  • ✅ Zero mandatory cloud dependencies — works fully offline via local runner
  • ✅ Published to AI Catalog — 4 agents + 1 flow available for any GitLab team to adopt
  • ✅ Reduced a process that takes $O(n \cdot 400)$ engineering hours (where $n$ = number of audits per year) to $O(1)$ per merge request

What we learned

  • Agent boundaries matter more than agent intelligence. A well-scoped agent with a clear input/output contract outperforms a clever agent with fuzzy responsibilities every time.
  • Compliance is a graph problem. A single code finding can implicate multiple controls across multiple frameworks simultaneously — modelling this as a many-to-many mapping was the right call.
  • The GitLab Duo Agent Platform is genuinely powerful for workflow-native AI. Triggering agents via MR mentions or reviewer assignment means zero friction for developers.
  • Graceful degradation is a feature, not a fallback. Building the system to work without GCP made it dramatically more adoptable for teams that can't or won't configure cloud credentials.

What's next for ComplianceBot Flow: From Manual to Instant Compliance

  • 🔄 Pre-merge blocking — fail pipelines that breach a configurable compliance score threshold (e.g., reject any MR scoring below 70/100)
  • 📊 Compliance drift dashboards — BigQuery-powered trend views showing how a team's compliance posture changes over sprints and quarters
  • 🧠 LLM-assisted remediation — instead of just flagging issues, suggest a specific code fix inline on the MR diff
  • 🌐 Framework expansion — adding NIST CSF, GDPR Article 32, and FedRAMP control mappings
  • 🤝 SIEM integration — push findings to Splunk, Datadog, or PagerDuty for real-time compliance alerting
  • 🏢 Enterprise policy engine — let organizations define custom controls in YAML and have ComplianceBot enforce them automatically across all repos

The vision: a world where shipping compliant code is the default, not the exception — and audits are a one-click export, not a quarterly crisis.

Built With

Share this project:

Updates