Inspiration

Every SRE team faces the same nightmare: a 3 AM page, frantic log-diving, and the dreaded post-incident report nobody wants to write. The average incident takes 47 minutes to diagnose — most of that time is spent correlating signals that humans can see but struggle to connect under pressure.

We asked: What if an AI agent could take the page, run through the diagnostic playbook instantly, and present a verified root cause with a remediation plan — all while keeping a human in the loop for safety?

That’s Aegis.


What it does

Aegis is an autonomous SRE agent that:

  • Receives incident alerts from Dynatrace via webhooks
  • Runs a structured 10-step reasoning loop: Classify → Gather → Correlate → Hypothesize → Verify → Propose → Await approval → Execute → Verify fix → Generate RCA
  • Streams its reasoning live to a real-time war-room UI so engineers can watch the agent think
  • Waits for human approval before any write action — no cowboy automation
  • Auto-generates a post-incident report once the fix is verified

It reduces Mean Time To Resolution (MTTR) from 47 minutes to under 60 seconds.


How we built it

Backend

Python 3.12 + FastAPI running on Cloud Run. The agent orchestrator (orchestrator.py) drives the 10-step loop, calling Gemini 2.5 Pro for reasoning via the google-genai SDK. A parallel Google ADK agent (adk_agent.py) provides a secondary investigation surface using FunctionTool wrappers.

Dynatrace MCP Integration

A custom MCP client (dynatrace_client.py) communicates with a Dynatrace MCP server using JSON-RPC 2.0. It registers five MCP tools:

  • problems-list
  • metrics-query
  • logs-search
  • traces-search
  • events-list

The client auto-selects between MCP, direct REST API, and a mock mode so the system works even without a live Dynatrace tenant.

Real-time streaming

An in-memory event bus publishes structured incident events. The API exposes them via SSE (/api/stream/{id}), consumed in the frontend using EventSource. Engineers see every reasoning step in real time.

Frontend

Next.js 16 App Router with TypeScript, Tailwind v4, Framer Motion, and a custom Dark Editorial design system:

  • OKLCH color tokens
  • Fraunces serif typography
  • Persimmon accent palette

Routes include:

  • Dashboard
  • Incident list
  • Per-incident war-room view

Infrastructure

  • Backend: Cloud Run via Cloud Build (cloudbuild.yaml)
  • Frontend: Vercel
  • License: MIT

Challenges we ran into

  • MCP protocol compliance — JSON-RPC 2.0 required careful handling and a fallback chain (MCP → REST → Mock)
  • Structured JSON from Gemini — occasional malformed outputs required validation + retry logic
  • SSE lifecycle management — keeping streams alive during multi-step + approval pauses was tricky
  • Next.js 16 breaking changes — required constant reference to official docs due to API shifts

Accomplishments that we're proud of

  • Built a fully autonomous incident-response loop with human-in-the-loop safety
  • Achieved real-time “agent reasoning visibility” in a live war-room UI
  • Designed a resilient multi-source observability connector (MCP + REST + fallback)
  • Reduced incident resolution time conceptually from minutes to seconds

What we learned

  • MCP is a powerful abstraction for tool use — once you build one connector, others are mostly wiring
  • Human-in-the-loop is not just safety — it’s a trust-building product surface
  • Structured, typed agent reasoning is significantly more debuggable than free-form chains

What's next for Aegis — Autonomous SRE Agent

  • Persistent storage (Firestore) for incident history and learning
  • Additional MCP connectors (Datadog, New Relic, PagerDuty)
  • Approval policies for auto-executing low-risk actions
  • Incident memory to improve hypothesis ranking using historical RCA data

Built With

  • cloud
  • cloud-build
  • cloud-run
  • dynatrace-mcp-(json-rpc-2.0)
  • fastapi
  • framer-motion
  • gemini-2.5-pro
  • google
  • google-adk
  • google-genai-sdk
  • next.js-16
  • pydantic-v2
  • python
  • radix-ui
  • react-19
  • recharts
  • secret
  • sse-(server-sent-events)
  • tailwind-css-v4
  • typescript
  • vercel
Share this project:

Updates