ORBIT Pipeline Rescue

• 

  ORBIT Pipeline Rescue: Evidence-driven CI recovery with approval controls, verified repairs, and safe escalation when fixes fail.

Inspiration

A failing CI pipeline creates pressure to make a quick fix. But an automated system that proposes a patch and immediately declares success can make a bad situation worse.

We built ORBIT Pipeline Rescue around a simple principle: a proposed repair is not a verified repair. We wanted to demonstrate a safer recovery workflow that distinguishes successful fixes from failed attempts and situations where the available evidence is insufficient.

What it does

ORBIT demonstrates a verification-gated pipeline recovery workflow built around Observe, Reason, Plan, Act, Verify, and Recover.

It demonstrates three important outcomes:

• Verified recovery: A generated candidate addresses a failing assertion, and the scenario reaches Resolved only after the verification test passes.
• Failed repair: A plausible candidate is applied, but verification still fails. ORBIT rejects the candidate and escalates the incident instead of reporting false success.
• Insufficient evidence: When the available evidence is ambiguous, ORBIT abstains from proposing a repair and escalates for further investigation.

The system records scenario evidence and makes the distinction between applying a change and verifying its outcome visible.

How we built it

We built ORBIT Pipeline Rescue with Python, FastAPI, SQLite, automated tests, and GitLab CI/CD.

The project includes a local demonstration environment, scenario-based testing, verification evidence, and a custom-flow definition at .gitlab/duo/flows/orbit-pipeline-rescue.yml.

The demonstration uses synthetic local fixtures. It is designed to show the recovery logic without connecting to a live pipeline or making remote repository changes. The repository includes automated tests and CI pipelines configured for Python 3.9 and Python 3.12.

Challenges we ran into

One of the main challenges was distinguishing a repair attempt from a successful recovery. A candidate can look reasonable and still fail verification. We therefore made the verification outcome central to the final incident status.

Another challenge was handling ambiguous evidence safely. Instead of forcing a repair when the evidence does not support a unique candidate, the system demonstrates abstention and escalation.

We also encountered a GitLab Duo Agent Platform integration limitation: the custom-flow definition is present and structurally validated, but registration and execution remain blocked in the project's current namespace. We have not represented the local demonstration as a live Duo execution.

Accomplishments that we're proud of

• Built a working local demonstration of three distinct recovery outcomes.
• Demonstrated a repair that reaches Resolved only after a passing verification test.
• Demonstrated rejection of an applied candidate when verification fails.
• Demonstrated abstention when the available evidence is ambiguous.
• Made the difference between candidate generation, approval, verification, and final status explicit.
• Published a public, MIT-licensed repository with GitLab CI/CD history and automated tests.

What we learned

We learned that reliable automation needs more than the ability to generate a plausible patch. It needs explicit controls, traceable evidence, and a verification step that can reject the system's own proposed solution.

We also learned that escalation is a valid outcome. A system that admits uncertainty is safer than one that always claims success.

What's next for ORBIT Pipeline Rescue

The next step is to complete and validate the GitLab Duo Agent Platform integration in an eligible project namespace, register the custom flow, and demonstrate an actual run with verifiable evidence.

We also want to extend the recovery workflow toward controlled integration with real CI pipelines, with explicit authorization, isolated execution, and verification gates before any recovery is accepted.

Our goal is to make automated pipeline recovery more trustworthy by ensuring that every claimed success is supported by evidence.

Built With

• agent
• ai
• automated
• ci/cd
• fastapi
• gitlab
• python
• sqlite
• testing