PipelineMedic

When CI goes red, PipelineMedic investigates, fixes it on a branch and opens a merge request with an evidence report, then stops. You review and merge. Deploys stay manual.

Comment

Inspiration

Every developer knows the red pipeline. Something fails, you open the job log, scroll through noise, find the one line that matters, and work out whether it is a real bug, a bad test or a config typo. The fix is often tiny. Finding it is what costs the time.

GitLab Duo can already fix pipelines in one click. I wanted something narrower and easier to trust: a helper that investigates, proposes, and then stops. It does not merge and it does not deploy. A human stays in charge of both.

What it does

PipelineMedic is a custom GitLab Duo flow. When a pipeline fails:

  1. You @mention the PipelineMedic bot on an issue. (Assign, Assign reviewer and Pipeline failed triggers also work.)
  2. It reads the failing job logs and the code, reproduces the failure by running the tests, and classifies it: test, lint, dependency, CI config or security.
  3. It creates a fix/pipeline-medic-* branch, makes the smallest fix, opens a merge request, and comments on the issue with the link.
  4. The MR description is a structured PipelineMedic report: pipeline, failing jobs, classification, root cause with quoted evidence, the fix, whether tests were touched, a confidence rating, and what the human should do next.
  5. You review the diff and merge. Deploys stay manual.

Supervised autonomy. The flow carries out the investigation, fix and MR on its own. You approve the outcome by merging.

Tests are the spec. The bot is told never to edit, delete or weaken tests to turn the pipeline green. If a test looks wrong and the code looks right, it escalates with a report and changes neither side. It is also told to escalate when it can't fix something safely or has used its two attempts.

Lifecycle stages covered

  • Plan: an issue starts the heal, and the bot posts its result there
  • Create: fix branch and merge request
  • Verify: lint and tests run again on every fix MR
  • Secure: SAST and secret detection run on every fix MR
  • Package: the pipeline builds the artifact on every fix MR
  • Release: a manual deploy gate that the bot is forbidden to touch

What I proved

  • Healed a wrong discount formula three times: a one-line fix each time, tests untouched.
  • Healed a real CI config error (an unquoted : in a YAML script line) that had turned my own main branch red.
  • Installed the flow from the AI Catalog on a second, unrelated project with no PipelineMedic files in its repo. It fixed a different bug (price + qty instead of price * qty) in a different codebase.
  • Given a deliberately wrong test, it did the arithmetic, saw that the code was right, flagged the test for a human, and changed neither the test nor the code. No merge request was opened.
  • The fix MR appeared within about a minute of the @mention in these runs.

How I built it

  • A custom flow in the GitLab Duo Agent Platform, written as flow registry YAML, running in a full coding environment with a toolset for reading logs, searching and editing code, running tests, creating branches and opening MRs.
  • A prompt with explicit rules: smallest fix, no test edits, keep security scans and required jobs, at most two attempts, escalate when unsure.
  • A demo app with a real GitLab CI pipeline: lint, test, SAST, secret detection, package, and a manual deploy job.
  • An optional CI guard template that fails merge requests that edit test files, and a CODEOWNERS file for sensitive paths.

Challenges

  • The chat agent could diagnose but not act. A custom agent in Duo Chat found the right root cause but couldn't reliably create a branch or an MR. Moving to a custom flow with a full coding environment solved it.
  • My own CI broke. A YAML line with an unquoted : made main red, and the flow fixed it. It was an accidental real-world test.
  • Linux vs Windows. A test glob that worked locally didn't expand on the CI runner.
  • Red-teaming my own guardrails. I tried to delete the guard jobs with an MR, and it merged cleanly. That showed that CI checks inside a repo are checks, not locks, so the project now recommends protected branches, required approvals and CODEOWNERS to make the human gate enforceable.

What I learned

Trust comes from limits, not capability. A bot that does less, shows its evidence and hands the decision to a human is easier to adopt than one that tries to do everything. I also learned to test my own claims before writing them down.

What's next

  • More languages and failure types beyond Node.
  • Installs from other groups and organizations.
  • Platform-level enforcement through GitLab policies.

Built With

Share this project:

Updates

Submission history