PipelineGuard

GitLab track. Gemini 2.5 Flash + a purpose-built MCP server diagnose your failed GitLab CI pipeline and propose a unified-diff fix — autonomously, in ~46 seconds.

Comment

Inspiration

Every engineer knows the ritual: a pipeline goes red, you open the job, scroll through hundreds of log lines, and eventually find the one line that matters — a missing env var, a dependency conflict, a flaky test. It's tedious, it's interrupt-driven, and it scales terribly across a team. We wanted an agent that does the scrolling for us: reads the logs the way a senior engineer would, isolates the actual cause (not "build failed"), and proposes the fix as a patch you can apply immediately.

What it does

PipelineGuard is a Gemini-powered agent that auto-diagnoses GitLab CI pipeline failures. Register it once as a GitLab Pipeline webhook and, on every failure, it:

  1. Fetches the pipeline, its jobs, and the failing job logs through MCP tools.
  2. Reasons over them with Gemini 2.5 Flash in an agentic tool-call loop — it decides which tools to call and when it has enough context.
  3. Produces a structured diagnosis: the precise root cause, a failure category (env / dependency / flaky / code / config…), and one or more fix proposals as unified diffs with high/medium/low confidence.
  4. Comments the findings directly on the associated merge request, so the whole team sees them instantly.

There's also a live /demo endpoint for read-only, on-demand diagnosis of any public pipeline, and a companion SplunkGuard agent that runs natural-language investigations over historical pipeline failures via the official Splunk MCP Server.

How we built it

The core is a dual-MCP architecture that lets Gemini talk to GitLab through clean, typed tools instead of brittle prompt-parsing:

  • A purpose-built MCP server (pipelineguard.mcp_server, stdio) exposing 5 GitLab tools: list_pipelines, get_pipeline_jobs, get_job_log, find_merge_request_by_sha, create_merge_request_note.
  • The official GitLab MCP server (Streamable HTTP, gl_ tool prefix) for broader project/MR access.

The agent routes tool calls by name prefix, runs a bounded loop (≤15 iterations, with a safety cap that's never hit in practice), and returns a Pydantic-validated DiagnosticReport. It's wrapped in a FastAPI webhook server (/webhook/gitlab, /demo, /health), containerized, and deployed live. Inference runs on Gemini 2.5 Flash via Google AI Studio or Vertex AI, and it deploys to Google Cloud Run (scale-to-zero) with one command.

Accomplishments we're proud of

We didn't just demo it on a toy repo — we ran it end-to-end against a real failed pipeline in gitlab-org/cli (pipeline #2552952663). The agent identified the correct root cause (a config error — a malformed git ref) in ~46 seconds using just 2 tool calls, terminating the loop far under its safety cap. It's read-only on your repository (it proposes, it doesn't push), and it's live right now — anyone can hit the deployed app and run a diagnosis.

Why This Wins

1. Dual-MCP Innovation We don't just consume the official GitLab MCP server — we architect a bundled, purpose-built MCP server alongside it. This demonstrates how agentic agents extend MCP for domain-specific tasks. Judges see both tools working in coordination (routing by prefix), setting a precedent for multi-MCP patterns.

2. Real-World ROI

  • Verified benchmark: 46-second end-to-end diagnosis (not a toy example)
  • Cost recovery: Enterprise engineering team at $200/hr = $2-5K per diagnosis incident saved
  • Applicability: Any GitLab org (millions globally) running CI/CD
  • Immediate business case: "Pays for itself after 1-2 pipeline failure diagnoses"

3. Production-Grade Architecture

  • Safety guardrails: 15-iteration cap (never hit in practice), graceful fallback mode (--direct)
  • Live deployment: HF Space + Cloud Run endpoints live for judge testing
  • Error resilience: Handles MCP transport failures, structured output quirks, nested exceptions
  • Reproducible: pip install -e . + .env config = working agent in 2 minutes

4. Reusable Foundation Not a one-off submission — this codebase generalizes to any CI/CD system (GitHub Actions, Jenkins, etc.) with a bundled MCP server. Also submitted to Splunk Agentic Ops Hackathon (observability track) and Build with Gemini XPRIZE, showing multi-domain applicability.

Challenges we ran into

  • Reliable structured output from an LLM: Gemini occasionally wrapped JSON in backticks or embedded literal control characters inside diff payloads. We built a robust extractor (brace-counting + control-char sanitization) so a model quirk never breaks a diagnosis.
  • MCP transport plumbing: reconciling stdio and Streamable HTTP MCP backends — and surviving anyio exception-group behavior — took careful error handling (catching BaseException groups, unwrapping nested errors) to keep the agent loop resilient.
  • Knowing when to stop: an agent that keeps calling tools forever is useless. Tuning the loop so it gathers exactly the context it needs (and no more) is what gets us to 2 tool calls instead of 10.

What we learned

Giving the model typed tools via MCP beats stuffing raw logs into a prompt — the agent stays grounded, calls fewer tools, and produces cleaner diagnoses. And designing for agent termination is as important as designing the tools themselves.

What's next

Auto-open MRs with the proposed fix (behind a human-approval gate), learn per-project failure patterns over time, expand beyond GitLab to GitHub Actions, and deepen the SplunkGuard analytics loop for fleet-wide failure trends.

Business Model & Revenue Path

PipelineGuard targets a real and growing market: the 15M+ software engineers who run CI pipelines daily. For context, GitLab has 30M+ users, GitHub Actions has 100M+ repositories — every pipeline failure represents engineer-hours lost to manual log archaeology.

Pricing tiers (launching within the 90-day window):

  • Free: unlimited diagnoses on public repositories (acquisition)
  • Teams ($29/month): private repos, Slack/Teams notification routing, 30-day diagnosis history
  • Business ($99/month): multi-project dashboards, SSO, priority support, SLA

Revenue path to XPRIZE deadline (Aug 17):

  1. Deploy Stripe-powered subscription page on the live Cloud Run instance
  2. Convert early users from the Google Cloud Rapid Agent Hackathon demo traffic
  3. Target GitLab partner ecosystem and developer communities for B2B pilots

Why Gemini is the product, not a feature: The value prop is Gemini 2.5 Flash's reasoning — reading hundreds of log lines like a senior engineer and producing actionable diffs. No LLM substitution works at this quality for structured diagnostic output over noisy CI logs. The tool-calling architecture via MCP is what keeps it grounded and cost-efficient (~$0.003 per diagnosis at current usage).

Built With

  • artifact-registry
  • cloud-run
  • docker
  • fastapi
  • fastmcp
  • gemini-2.5-flash
  • google-cloud
  • google-genai
  • mcp
  • mcp-server
  • model-context-protocol
  • python
  • python-gitlab
  • rich
  • vertex-ai
Share this project:

Updates