• 

  ui

• 

  incident response

• 

  gitlab review response

FixLoop: Evidence-Backed Security Remediation with GitLab Duo + Orbit

Inspiration

Security scanners can identify suspicious code, but a finding alone does not tell a developer what to fix, how to verify the fix, or whether the available evidence actually supports marking an issue as resolved.

I wanted to explore how AI agents could help close that gap. Instead of building another tool that simply reports vulnerabilities, I built FixLoop around an evidence-first workflow: investigate a finding, examine the relevant code and tests, propose a targeted remediation, verify the available CI evidence, and leave the final decision to a human.

The goal is to make security reviews more actionable without allowing an AI agent to make unsupported claims or take uncontrolled production actions.

How I Built It

FixLoop builds on my existing MIT-licensed VibeSecure security-scanning project. For this hackathon, I extended that foundation with a GitLab-focused workflow integrating GitLab Duo Agent Platform, GitLab Orbit, repository evidence, and CI verification.

The hackathon-specific implementation includes:

• GitLab Duo flow: A custom FixLoop Evidence Review flow that investigates a merge request and produces an evidence-based security review.
• GitLab Orbit integration: The flow discovers available Orbit commands and the graph schema before querying for relationships between the merge request, pipeline, jobs, and security findings.
• Independent evidence checks: GitLab API results are used to corroborate current merge-request and pipeline information instead of blindly trusting potentially stale graph data.
• Regression testing: A synthetic authorization example demonstrates how a vulnerable baseline and a corrected policy can be evaluated through tests.
• Human-controlled remediation: The flow can publish a review note, but code changes, merging, deployment, and incident-response actions remain human responsibilities.
• Operator guidance: An incident-response runbook and a reproducible demo help explain the workflow and its limitations.

What Makes FixLoop Different?

The central idea is that a security conclusion should be tied to evidence.

FixLoop distinguishes directly observed facts from hypotheses, reports missing or inconclusive evidence, and avoids treating an empty graph query as proof that no vulnerabilities exist. It also distinguishes a successful synthetic regression test from verification of a real production system.

This approach makes the review more useful to developers and makes the agent's limitations visible instead of hiding them behind a confident AI-generated answer.

Challenges I Faced

One of the biggest challenges was aligning the custom flow with GitLab's actual configuration schema. The editor rejected an unsupported tool-approval setting, while an existing safety test still expected that setting to be present. This caused CI failures.

I resolved the mismatch by removing the unsupported assertion and updating the flow instructions and documentation to describe the actual safety boundary honestly. The subsequent pipeline passed, with both the security-remediation demo and safety-contract jobs succeeding.

Another challenge was interpreting Orbit evidence responsibly. Graph queries can return incomplete or empty results, so I designed the workflow to corroborate important merge-request and pipeline facts through GitLab's API and explicitly report evidence gaps.

I also learned that a working flow configuration is not enough: a credible demonstration requires an actual agent session, observable tool activity, test results, and a clear explanation of what has—and has not—been verified.

What I Learned

Building FixLoop taught me that useful AI security tooling is not just about generating patches. It also requires reliable tool integration, regression tests, evidence provenance, careful handling of uncertainty, and clear boundaries between agent recommendations and human decisions.

I learned to treat CI failures as feedback about the implementation contract, verify agent behavior through real sessions, and document limitations rather than overstate results.

Current Status and Limitations

The custom GitLab Duo flow completed a real session against merge request !1. Orbit schema discovery and graph queries succeeded, GitLab API evidence corroborated the merge request and CI jobs, and the flow published a review note.

The synthetic authorization test demonstrates a narrow regression scenario; it does not establish production security. The Orbit finding query returned no rows, which does not prove the absence of vulnerabilities. The running scanner builds on the pre-existing VibeSecure foundation, and a complete end-to-end production scanner run has not been verified as part of this demonstration.

The flow's accepted YAML does not establish a separate per-tool approval gate for publishing review notes. FixLoop therefore does not claim that this configuration independently enforces such a gate.

What's Next?

My next priorities are to record a concise, reproducible demonstration; strengthen the verification workflow; review the remaining deployment and dependency limitations; and improve the path from a supported finding to a tested remediation that a human can confidently review.

FixLoop's principle is simple: don't just suggest a fix—show the evidence, verify what can be verified, and keep consequential decisions accountable.

Built With

• ai-agents
• application-security
• automation
• ci/cd
• cybersecurity
• devsecops
• gitlab
• gitlab-duo
• gitlab-orbit
• llm
• opensource
• python
• security-testing