Compliance Evidence Packager

Turn 6 weeks of manual SOC 2 audit prep into 15 minutes of automated evidence with a Reactive GitLab AI Agent.

Comment

🔐 Compliance Evidence Packager - Hackathon Submission

Inspiration

Compliance is the "hidden tax" on engineering speed. Traditional audits for SOC 2, ISO 27001, HIPAA, and FedRAMP force developers to stop building for weeks to manually hunt for merge request approvals, pipeline logs, and security scans. This "annual panic" costs enterprises hundreds of thousands in engineering hours.

We built the Compliance Evidence Packager because compliance should be a byproduct of excellent engineering, not an administrative burden. By living natively within the GitLab ecosystem, our Agent transforms manual evidence gathering into a continuous, streaming data engine.

What it does

The Compliance Evidence Packager is a Reactive AI Agent built on the GitLab Duo Agent Platform. It continuously monitors the SDLC and takes definitive action:

  1. Continuous Monitoring: Auto-analyzes every Merged MR and Pipeline run against 4 major frameworks (SOC2, ISO27001, HIPAA, FedRAMP).
  2. Audit-Ready PDF Generation: Compiles metrics into enterprise-grade reports with executive summaries and detailed appendices.
  3. Auto-Remediation Ticketing: If a control fails (e.g., a "Rubber Stamp" approval is detected), the Agent automatically creates a formatted GitLab Issue with remediation steps and assigns it to the team.
  4. Multi-Modal Interaction: Available via GitLab Duo Chat for on-demand queries and GitLab AI Flows for automated MR reviews.

How we built it

To move beyond a simple API script and reach Enterprise-Grade reliability, we implemented several key architectural innovations:

♾️ Infinite Memory Scaling (Generators)

Auditing repositories with 100,000+ merge requests often causes standard scripts to crash due to OOM (Out of Memory). We built a single-pass streaming architecture using Python yield generators.

The peak memory footprint is mathematically guaranteed to stay constant: $$M_{peak} \approx 30 \text{ MB} \text{ (where } \text{data size } \to \infty )$$ Even with 1 million MRs, the Agent uses less RAM than a Chrome tab.

🔄 Elastic Crash Recovery (SQLite Caching)

Network failures on page 800 of 1,000 shouldn't mean restarting from page 1. We built a local SQLite persistence layer that intercepts the API loop. If interrupted, the Agent instantly resumes from the last successfully cached page.

🔍 The Rubber Stamp Detector (Heuristics)

Instead of binary "Check-the-box" compliance, our Evidence Collector uses heuristics to detect superficial reviews. The Rubber Stamp Risk is calculated as: $$\Delta t = t_{merged} - t_{created}$$ If $\Delta t < 15\text{ minutes}$, the Agent flags the MR as a compliance risk, even if it has the required number of approvals.

Challenges we ran into

  • API Rate Limits: Managing the "Thundering Herd" problem when scanning massive audit periods (1+ years). We solved this with exponential backoff using the GitLab Retry-After headers.
  • Permission Granularity: Designing the Agent to handle missing data gracefully (e.g., if it doesn't have Developer access to read security scans) without failing the entire audit.

Accomplishments that we're proud of

  • 100% API-to-Control Mapping: We successfully mapped technical GitLab API fields (approvals, upvotes, pipeline status) to rigorous legal control requirements (CC6.1, CC7.2, CC8.1).
  • GitLab Duo Integration: Successfully publishing the agent to the GitLab AI Catalog, making it available via Model Selection and AI Flows.
  • Rock-Solid Resilience: Verified <30MB RAM usage even when streaming live data from the largest public GitLab projects.

What we learned

  • Deep Agent Integration: We learned that a great AI agent isn't one that just talks, but one that acts on its environment by creating issues and generating artifacts.
  • Scaling Python: Mastering the use of generators and local persistence for extreme data processing.

What's next for Compliance Evidence Packager

  • 📊 Web Dashboard: A real-time visualization of compliance scores across entire GitLab groups.
  • 🤖 ML Anomaly Detection: Using statistical analysis to detect drift in deployment frequency and approval patterns: $$\text{Score}{compliance} = \sum{i} w_i \cdot \text{Control}_i + \text{AnomalyPenalty}$$
  • 🌐 More Frameworks: Expanding to PCI-DSS, GDPR, and NIST 800-53.

Built With

  • flask
  • gitlab-api
  • gitlab-ci-cd
  • gitlab-duo-agent-platform
  • gitlab-webhooks
  • pytest
  • python
  • python-dotenv
  • pyyaml
  • reportlab
  • requests
  • rest-api
  • sqlite
  • streaming-architecture
Share this project:

Updates