DuoGuard

Meta Agent. Audits AI agent configs on GitLab Duo for security & compliance, with BigQuery audit trails and Looker Studio dashboards.

Comment

Inspiration

What if an AI agent could audit other AI agents?

AI agents are multiplying across GitLab projects — reading files, writing code, running commands, creating merge requests. But who's watching the watchers?

In enterprise data management, you don't give employees admin access and hope for the best. You enforce registries, access controls, audit trails, and lifecycle policies. With 15+ years in Data Governance across telecom and fintech, I recognized the gap immediately: nobody was applying these proven governance principles to AI agents.

GitLab Duo makes it easy to create agents. It doesn't tell you if those agents have god-mode access, dangerous tool combinations, or zero safety guardrails. That's the problem DuoGuard solves — as a meta-agent: an AI agent purpose-built to audit other AI agents.

What it does

DuoGuard is a meta-agent that audits other AI agent configurations inside GitLab. You invoke it from Duo Chat, it scans your project's agents, and scores each one A–F for governance risk.

It evaluates four governance dimensions:

  • Registry — Are agents documented, owned, and inventoried?
  • Access — Do agents follow least-privilege? Are dangerous tool combinations present?
  • Audit Trail — Is there a git history of configuration changes and review gates?
  • Lifecycle — Are there stale or orphaned agents nobody is maintaining?

DuoGuard creates a scored GitLab Issue with findings and recommendations, and logs every audit to Google BigQuery for historical tracking via Looker Studio dashboards.

One conversation. Every agent scored. A to F.

How we built it

  • GitLab Duo Agent Platform — Custom Agent with a multi-step Custom Flow that orchestrates the full audit pipeline
  • Anthropic Claude — Powers the governance analysis natively through GitLab Duo
  • Google BigQuery — Stores the audit trail via a CI/CD job triggered post-audit
  • Looker Studio — Connects to BigQuery for governance health dashboards and trend analysis
  • Scoring engine — JSON-based rules for deterministic checks (dangerous combos, tool counts) combined with LLM-powered analysis for contextual evaluation

Challenges we ran into

  • GitLab Duo flows don't support automatic file-change triggers yet — we used mention and pipeline event triggers as workarounds
  • No direct BigQuery integration from agent flows — we built a separate CI/CD job triggered by the duoguard-audit label on issues
  • Custom agents are configured via GitLab UI, not YAML files — we maintained YAML as source-of-truth documentation while configuring through the interface
  • Balancing deterministic scoring rules with LLM flexibility — some checks need exact pattern matching (god-mode combos), others need contextual understanding (safety guardrails in prompts)

Accomplishments we're proud of

  • A working meta-agent that audits what no one else is auditing — agent permissions, dangerous tool combinations, and lifecycle hygiene inside GitLab
  • End-to-end audit pipeline: trigger → scan → score → issue → BigQuery → dashboard, fully automated
  • Four governance dimensions that translate proven Data Governance principles (registries, access control, audit trails, lifecycle) into the AI agent world
  • Deterministic + LLM hybrid scoring — consistent, reproducible results with the nuance to understand context
  • The same config always gets the same score. You can audit the auditor.

What we learned

The principles of Data Governance translate directly to AI Agent Governance. Registries, access control, audit trails, lifecycle management — these aren't new concepts. They're proven practices from decades of enterprise data management, now applied to a new challenge: autonomous agents operating inside your development platform.

The biggest insight: governance shouldn't slow teams down — it should run in the background and only surface when something needs attention. A meta-agent is the right delivery mechanism: it speaks the same language as the agents it audits, runs where they run, and requires zero external tooling.

"You wouldn't give a new employee admin access on day one. Why do it with AI agents?"

What's next for DuoGuard

  • Automatic triggers when agent configuration files change (pending GitLab platform support)
  • Action layer — runtime behavior monitoring of what agents actually do during sessions
  • Multi-project governance across GitLab groups and organizations
  • Governance policy templates for enterprise compliance frameworks (SOC2, ISO 27001)
  • Trend analysis — detect governance drift over time and alert before risks become incidents

Built With

Share this project:

Updates