Apex — Self-Healing Pipeline Orchestrator

Pipeline broke? Apex fixed it. Five AI agents diagnose the root cause, push a clean fix branch, and hand you a ready-to-merge MR — zero toil for developers.

Comment

The Problem I Kept Running Into

I was shipping features every day, but the real friction wasn't the code — it was everything around it.

Merge conflicts piling up because the team moves fast. Security vulnerabilities surfacing mid-sprint because of SOC2 requirements I can't ignore. And every time a pipeline failed, the drill was the same: drop what I'm doing, switch branches, investigate, patch, push — and somehow not break what I already had in flight.

Worse, the "automated" solutions I tried polluted my working branch. Suggested fixes that never got picked up. Patches that conflicted with my in-progress changes. I wanted my original code to stay clean — but I still needed a way to test, fix, and move forward without losing momentum.

What I Built

Apex is a self-healing pipeline orchestrator built on GitLab Duo Agent Flows.

The idea is simple: when your pipeline fails, you assign Apex as a reviewer. That's it. You walk away.

Behind the scenes, five AI agents go to work:

  • Sentinel — gathers full MR context, logs, and environment state
  • Forensic — runs 4-layer root cause analysis (what failed → why → recurrence history → security patterns)
  • Surgeon — generates a targeted fix and pushes it to a separate branch off yours — never touching your original code
  • Triage — classifies the issue, scores duplicates, creates a tracked work item
  • Herald — posts a full diagnostic report with time-saved metrics and escalates security findings

The fix lands as a clean MR targeting your branch. You review it, merge it, done. No context switch. No branch pollution. Your original work stays exactly as you left it.

What I Learned

Building Apex taught me that the real problem with broken pipelines isn't the failure — it's the interruption. Every time a developer has to stop and investigate, they lose flow. The fix isn't just automation; it's automation that respects the developer's context.

The hardest part was making sure Apex never overwrites in-progress work. The Surgeon agent had to be explicitly constrained: always target the developer's feature branch, always create a new fix branch on top, never rewrite history.

I also learned a lot about multi-agent orchestration — specifically how to hand off state cleanly between agents so each one gets exactly what it needs without re-running expensive operations.

Challenges

  • Merge conflict resolution — teaching the Surgeon to perform a 3-way LLM merge when the fix branch diverges from the developer's branch
  • Security archaeology — scanning git history for vulnerability recurrence without blowing up on large repos
  • Deduplication — semantically scoring existing issues before creating new ones to avoid GitLab noise
  • Developer trust — the entire UX had to feel like a colleague handing you a clean PR, not a bot spamming your inbox

Built With

  • claude-sonnet-4.6-(anthropic)
  • gitlab-ci/cd
  • gitlab-duo-agent-flows
  • gitlab-rest-api
  • python-3.11
  • yaml
Share this project:

Updates