CI Repair Review

• 

  Recorded GitLab fixture: six original tests pass, two failures recover, and maintainer approval is retained.

Demonstration sources: recorded GitLab Duo and CI execution for the repository-owned queue fixture; a separate controlled local replay for the removed-test case.

Inspiration

Small-team maintainers need a clear way to review both a repair and the original checks that give that repair meaning. CI Repair Review brings repair results, original test checks, and supporting evidence into one review for the maintainer.

What it does

The deterministic reviewer compares original test identities, protected fixture files, and before-and-after JUnit results. It identifies recovered failures, retained tests, and changes that need attention, then exports the decision with its supporting evidence. A custom read-only GitLab Duo agent explains the recorded pipeline, producer job, and review artifacts while keeping the maintainer in control.

How we built it

We built the reviewer and command-line tools with the Python standard library. A Git-revision evidence producer runs the repository-owned queue fixture at a selected baseline and candidate commit, saving JUnit, a source diff, a review report, and a run manifest. The recorded demonstration executes this fixture within the same GitLab project, with its execution scope stated in the manifest. GitLab's existing Fix CI/CD Pipeline flow supplies the repair stage; our contribution is the deterministic evidence review and its maintainer-facing explanation. Our custom agent made five read-only tool calls to retrieve the pipeline, exact producer job, and three job artifacts.

Challenges we ran into

We separated repair findings from execution references so the evidence remains inspectable and the core decision stays visible. The custom agent reads the exact job artifacts and preserves the maintainer approval step.

Accomplishments

GitLab Duo's repair flow created draft merge request one with a one-line queue-planning fix. The repaired candidate pipeline passed, including the queue target and the evidence-producing job. The resulting report records all six original fixture tests passing and both earlier fixture failures recovered. The fixture test file and fixture CI configuration retain their selected baseline contents. The custom agent returned CONTROLLED_DEMO_REVIEWED, explained the fixture results, and preserved the core ready_for_review decision with automatic approval disabled. In a separate local validation case, all five remaining tests pass after one original test is removed, and the reviewer still identifies the missing case and changed test file.

What we learned

Preserving test identity makes the original test contract visible alongside a passing-test count. Recording the selected baseline, candidate, exact job, and report fingerprint gives a maintainer concrete references for reviewing the result.

What's next for CI Repair Review

We plan to extend the review workflow to more project-specific test contracts and a separately controlled verifier. The next demonstrations will keep the repair, evidence producer, and maintainer decision connected through explicit run references.

Built With

• css
• gitlab
• gitlab-ci/cd
• gitlab-duo-agent-platform
• html
• javascript
• python