AutoSRE — Autonomous CI Failure Recovery Agent

AutoSRE — Autonomous AI Agent for Software Failure Recovery

Inspiration

Every developer knows the pain: a CI pipeline fails at 2am, Slack lights up, and someone has to manually dig through logs, find the bug, push a fix, and re-trigger the build. For SRE teams managing dozens of microservices, this is a daily reality. We asked: what if an AI agent could handle the entire recovery loop — without waking anyone up?

What It Does

AutoSRE is an autonomous AI agent that closes the entire software failure loop:

Detects failed CI/CD pipelines via GitLab API
Fetches failure logs and affected source files automatically
Analyzes root cause using Gemini 2.5 Pro on Google Cloud Vertex AI
Generates a minimal, targeted code fix
Creates a new Git branch and commits the fix
Opens a Merge Request with a structured explanation — for human approval

The result: a broken pipeline triggers the agent, and within minutes a reviewed, ready-to-merge fix appears in GitLab — no human debugging required.

Demo

Real run output:

Detected failed pipeline #2534952310, job run_tests
Gemini diagnosed: missing requests module in CI + incorrect status assertion
Created branch autosre-fix-14427681587
Committed fix, opened Merge Request !1
Pipeline on the fix branch passed

How We Built It

Stack:

Google Cloud Vertex AI — Gemini 2.5 Pro for root cause analysis and fix generation
Google Cloud Run — hosts the intentionally buggy Flask microservice
GitLab CI/CD — pipeline detection, log retrieval, branch/commit/MR management via REST API
Python — orchestrates the full agent loop end-to-end

Agent Architecture: Failed Pipeline ↓ GitLab API → fetch logs + source files ↓ Gemini 2.5 Pro → structured JSON (root_cause, fix_description, fixed_file) ↓ GitLab API → create branch → commit fix → open MR ↓ Human reviews & merges

The agent is deliberately designed with a human-in-the-loop gate at the MR stage. The AI proposes; the engineer approves. This makes it safe for real production use.

Challenges

Model endpoint routing on Vertex AI — getting the correct regional endpoint and model string for Gemini 2.5 Pro required careful debugging
Structured output reliability — prompting Gemini to return valid JSON consistently required explicit schema instructions and fence-stripping logic
Fix quality vs. fix scope — instructing the model to make minimal changes (rather than rewriting everything) took prompt iteration

What We Learned

Gemini 2.5 Pro is remarkably capable at multi-file code analysis when given full context (logs + source)
The GitLab REST API is powerful enough to implement a full git workflow programmatically
Human-in-the-loop design isn't just a safety fallback — it's the feature that makes AI agents trustworthy in production