ShipSafe Release Orchestrator

Inspiration

Software developers often spend valuable time investigating failed CI/CD pipelines, understanding test failures, reviewing security results, and deciding whether a change is ready for release. AI agents can help with these tasks, but proposed fixes must be verified before they are trusted.

We built ShipSafe Release Orchestrator to explore how GitLab Duo Agent Platform can assist developers with release investigation while keeping human reviewers in control.

What it does

ShipSafe is an AI-assisted release review workflow that helps teams investigate pipeline failures and collect evidence for release decisions.

Key capabilities include:

• Pipeline investigation: Examines pipeline and job results to identify failures.
• AI-assisted diagnosis: Investigates available logs and suggests possible fixes.
• Recovery verification: Uses a separate successful CI pipeline as evidence that the tested revision passed configured checks.
• Security review: Brings together available Static Application Security Testing (SAST), Dependency Scanning, and Secret Detection results.
• Human-controlled decisions: Distinguishes a successful pipeline from a fully approved release and leaves unverified approvals or checks outstanding.

How we built it

We built ShipSafe using the GitLab Duo Agent Platform, including a custom AI agent named ShipSafe Release Orchestrator and a workflow named ShipSafe Release Review.

The project uses a .NET solution with application and test projects, integrated with GitLab CI/CD. The pipeline includes application build, automated tests, SAST, Secret Detection, and Dependency Scanning.

The agent can investigate a failed pipeline, analyze the available evidence, propose a code correction, create a new branch, and open a draft merge request for human review.

We also tested the workflow against an actual failed test pipeline and a separate successful pipeline. This helped demonstrate the difference between diagnosing a failure and independently verifying a proposed recovery.

Challenges we ran into

One challenge was distinguishing a genuine application defect from a deliberately failing test. The agent needed to examine the test assertion and its expected behavior rather than assuming that every failed test meant the application code was incorrect.

Another challenge was that the agent's own execution environment did not have the .NET SDK available to run the tests. We therefore used GitLab CI pipeline results as separate verification evidence instead of claiming that the proposed fix had been proven by the agent itself.

We also needed to avoid treating a successful pipeline or an empty security findings list as a guarantee that a release was completely safe and approved.

Accomplishments that we're proud of

• Built a custom AI agent and release review workflow in GitLab Duo Agent Platform.
• Demonstrated diagnosis of an actual failed test pipeline.
• Demonstrated a proposed correction and draft merge request for human review.
• Verified a separate pipeline in which all five configured jobs passed, including eight tests with zero failures.
• Demonstrated security scan results reporting zero vulnerabilities across SAST, Dependency Scanning, and Secret Detection.
• Designed the workflow to support human judgment rather than automatically approving every release.

What we learned

We learned that useful AI-assisted software engineering requires more than generating a plausible fix. The system must distinguish observations from assumptions, use independent test results as verification evidence, and communicate uncertainty clearly.

We also learned that passing tests and reporting zero vulnerabilities are important signals, but they do not guarantee that software is free of defects or security weaknesses.

Most importantly, release readiness should consider available technical evidence and required human approvals separately.

What's next for ShipSafe Release Orchestrator

Our next steps are to strengthen merge-request approval verification, expand testing across more pipeline failure scenarios, improve evidence traceability, and make release recommendations more comprehensive.

Our goal is to help engineering teams spend less time investigating routine pipeline failures and more time making informed, evidence-based release decisions.

Built With

• .net
• agent
• ci/cd
• duo
• gitlab
• platformgitlab
• sast
• xunit