skill-auditor

• 

  6 GitLab Duo agents audit every AI skill in your repo for prompt injection, data exfiltration & supply chain attacks.

Inspiration

While exploring the GitLab Duo Agent Platform, I came across the "Agent Skills in the Wild" paper (SkillScan, 2025) — a large-scale study of 31,132 public AI agent skills. The finding stopped me cold: 26.1% of public skills contain security vulnerabilities, ranging from prompt injection to credential theft to ransomware delivery. Yet developers install skills from public marketplaces with no security review whatsoever. The problem wasn't being discussed — and no tooling existed to address it at the point of adoption.

What I Learned

• Traditional SAST tools have 0% recall on prompt injection — the most dangerous vulnerability class requires semantic LLM analysis, not pattern matching.
• Supply chain and data exfiltration attacks co-occur 81% of the time — this correlation is detectable and actionable if agents share context.
• The scanner itself is an attack surface. A sophisticated skill can embed hidden instructions designed to fool the security agent auditing it — a meta-injection attack. Defending against it requires treating all skill content as untrusted input by design.
• GitLab Duo Custom Flows are powerful but currently sequential-only — which forced a deliberate architectural decision to turn a constraint into a feature (cascading context between agents).

How I Built It

I designed a 6-agent GitLab Duo Custom Flow grounded in the SkillScan vulnerability taxonomy (14 patterns across 4 categories):

1. Scout Agent — discovers all SKILL.md files and bundled scripts in the target repo using list_repository_tree and get_repository_file.
2. Prompt Injection Agent — detects P1–P4 and E4 patterns using semantic LLM analysis (the only method that works).
3. Data Exfiltration Agent — detects E1–E3, cross-referencing PI findings to flag coordinated attacks.
4. Privilege Escalation Agent — detects PE1–PE3, flagging credential theft when combined with exfiltration patterns.
5. Supply Chain Agent — detects SC1–SC3, applying the 81% SC→DE co-occurrence rule during live analysis.
6. Reporter Agent — aggregates all findings using a union strategy, produces 0–10 risk scores, and posts a structured markdown audit report as a GitLab issue comment.

Each specialist agent begins with a Meta-Injection Guard in its system prompt — making any skill that tries to fool the scanner automatically flag itself as a P1/P2 violation.

Challenges

• Sequential-only routing in GitLab Flows — the ideal architecture is parallel fan-out. I kept the specialist-per-category design and made it parallel-ready: all four specialist agents take skills_content independently from the Scout, so only the router block needs updating when parallel routing ships.
• Defending the auditors themselves — realizing the scanner is itself a target required adding meta-injection guards to every agent's system prompt, not just the input validation layer.
• Calibrating the union strategy — balancing false positives (too sensitive) vs. missed detections (too lenient) required defining confidence thresholds: ≥0.6 to accept a finding, ≥0.8 to overturn a static flag via LLM reasoning.

Built With

• antigravity
• gitlab-duo-agent-platform
• gitlab-flows
• remotion