Rootiq — DevOps Intelligence Agent (AI Incident Responder)

AI agent that detects CI pipeline failures, finds the root cause, and opens a patch MR automatically.

Comment

Inspiration Every pipeline failure triggers the same 40-minute manual loop: open four tabs, hunt through 10,000 lines of logs, cross-reference a Jira ticket from three weeks ago, write a GitLab issue you could have written in 10 seconds. This happens dozens of times a week at every engineering team. We wanted to build the agent that closes that loop before the engineer opens a terminal.

What it does Rootiq is a DevOps Intelligence Agent. The moment a GitLab CI pipeline fails, it wakes up, finds the root cause, and fixes it — automatically.

It calls the GitLab MCP server to fetch failed job logs, searches Elasticsearch for error patterns, queries MongoDB for similar past incidents, enriches context via Fivetran (Jira, PagerDuty, DB metrics), calls the LLM for a structured root-cause analysis with a confidence score, then creates a GitLab issue and — if confidence is ≥ 75% — opens a patch MR. All 10 steps run in under 10 seconds.

Live result: 91% confidence analysis, root cause identified, GitLab issue created at gitlab.com/shakeebiqbals/rootiq/-/work_items/12, end-to-end in 8.9 seconds.

How we built it Node.js 22 ESM with Express handling GitLab Pipeline Hook webhooks via a serial queue — returns 200 immediately so GitLab never times out.

The core is a 10-step DevOpsAgent orchestrator using Promise.allSettled throughout, so any single tool failure never aborts the full analysis.

GitLab MCP server (gitlab.com/api/v4/mcp) is called via JSON-RPC 2.0 HTTP transport for get_pipeline_jobs, get_job_log, create_issue, and create_merge_request. Elasticsearch handles real-time log pattern extraction. MongoDB stores the incident history with error-pattern overlap scoring for similarity search. Fivetran pulls enriched external context in parallel. Arize Phoenix receives OTLP spans for every agent run — input, output, confidence, token count, latency.

The LLM layer (gemini-client.js) is a thin abstraction designed to swap between Claude (development) and Gemini via Vertex AI (production), with the full production config in cloud/agent-builder.yaml. The confidence gate at 75% prevents noisy auto-patch MRs from eroding engineer trust.

The React dashboard at rootiq-delta.vercel.app shows the live incident monitor with the agent thinking ticker, confidence meter, partner badges, and direct links to GitLab issues and Arize traces.

Challenges we ran into Three hard problems worth noting.

First: GitLab's MCP server requires a GitLab Ultimate namespace with Duo enabled. The fallback architecture — MCP first, REST second, mock third — meant the agent could run end-to-end at every stage of development without blocking on credentials.

Second: Promise.all anywhere in the 10-step flow would let a single 404 abort the entire analysis. Every parallel operation uses Promise.allSettled with inline fallbacks. This took two debugging sessions to get right.

Third: the LLM confidence gate design. Auto-patching at low confidence would create noisy, incorrect MRs that engineers would stop trusting within a week. Getting the system prompt to produce honest confidence scores — penalising itself when evidence is insufficient — required careful prompt engineering and a fallback analysis path when JSON parsing fails.

Accomplishments that we're proud of The agent runs a full 10-step analysis and creates a live GitLab issue in under 10 seconds with zero manual intervention.

The MongoDB similarity matching — scoring past incidents by error pattern overlap — means the agent genuinely improves over time. Repeat failures resolve faster because the agent has seen them before.

The architecture degrades gracefully at every layer. No live credentials required to run the full demo. Elastic down? Mock patterns. MongoDB unreachable? Mock history. Arize not configured? Local logging. The agent always completes a full run and always creates a GitLab issue.

Every single agent decision is auditable in Arize Phoenix — not just the application, but the agent itself. Confidence drift, token usage, latency, and LLM input/output are all traced per incident.

What we learned MCP is the right abstraction for tool-using agents — but the credential and namespace requirements for partner MCP servers mean robust fallback architecture is non-negotiable for development.

Promise.allSettled over Promise.all is not optional when you have 5 external tools in a single agent run. One network hiccup should degrade gracefully, not abort the analysis.

Confidence gates matter more than accuracy. An agent that auto-patches at 60% confidence will train engineers to ignore it within a week. The 75% gate with explicit evidence requirements is what makes the agent trustworthy rather than just impressive in a demo.

Arize as a meta-layer — observing the agent itself rather than the application — gives you something none of the other integrations provide: the ability to audit why the agent concluded what it did, and to measure whether that conclusion was right.

Share this project:

Updates