Evoo

EVOO is an AI SRE agent that autonomously detects, fixes, and learns from prod incidents.It scores every fix, remembers what works, and gets smarter each time ,turning 90s outages into 15s recoveries.

Comment

🫒 EVOO — Project Story

Inspiration

It started with a 3 AM page.

A production service was down. The on-call engineer woke up, rubbed their eyes, SSH'd into the server, and started the familiar ritual: check logs, check metrics, try a restart, wait, check again. Forty-five minutes later, the service was back. The next morning, the same engineer wrote a postmortem that looked almost identical to the one from two weeks ago.

We asked ourselves a simple question: Why doesn't incident response get better over time?

Humans learn from experience — but our systems don't. Every incident is treated as if it's happening for the first time. The knowledge of what worked lives in Slack threads, in postmortems nobody reads, in the heads of engineers who eventually leave.

We were inspired by how reinforcement learning agents master games — not by being told the rules, but by playing thousands of rounds, scoring their performance, and gradually discovering optimal strategies. What if we could build an agent that does the same thing for infrastructure operations?

That's how EVOO was born: an Evolutionary Operations Optimizer — an AI agent that doesn't just respond to incidents, but learns from every single one.

What We Learned

1. The Exploration-Exploitation Dilemma Is Real

The most fascinating challenge was balancing exploration (trying new strategies to discover better ones) with exploitation (using strategies we already know work well). This is the classic multi-armed bandit problem from reinforcement learning.

We implemented an $\varepsilon$-greedy strategy with decay:

$$P(\text{explore}) = \varepsilon_t = \max\left(\varepsilon_{\min},\ \varepsilon_0 \cdot \gamma^t\right)$$

Where:

  • $\varepsilon_0 = 0.3$ (initial exploration rate — 30% of the time, try something new)
  • $\gamma = 0.95$ (decay factor — reduce exploration by 5% each cycle)
  • $\varepsilon_{\min} = 0.05$ (minimum exploration — always keep 5% for discovery)
  • $t$ = number of completed cycles

Early on, EVOO explores aggressively. By cycle 10, it's exploiting proven strategies 95% of the time — but never stops learning entirely.

2. Reward Design Is Everything

We learned that how you score success determines what the agent optimizes for. A naive reward function ("did the service come back?") produces an agent that restarts everything — fast but expensive and disruptive.

Our multi-factor reward function captures what a real SRE cares about:

$$R = R_{\text{base}} + R_{\text{improve}} - R_{\text{penalty}}$$

Where:

$$R_{\text{base}} = \begin{cases} 100 & \text{if service restored} \ 0 & \text{otherwise} \end{cases}$$

$$R_{\text{improve}} = 0.1 \cdot \Delta_{\text{latency}} + 100 \cdot \Delta_{\text{availability}} + 0.3 \cdot \Delta_{\text{cpu}} + 0.3 \cdot \Delta_{\text{memory}}$$

$$R_{\text{penalty}} = 0.5 \cdot t_{\text{recovery}} + 0.2 \cdot C_{\text{infra}} + 50 \cdot e_{\text{after}}$$

This means EVOO doesn't just fix things — it learns to fix things fast, cheaply, and cleanly.

We then added an LLM-as-judge layer that adjusts the numerical reward by $\pm 20$ points based on qualitative assessment: Was the strategy appropriate? Were the actions efficient? This hybrid approach (formula + LLM judgment) produces more nuanced learning than either alone.

3. Safety Guardrails Are Non-Negotiable

An autonomous agent that can restart services and scale infrastructure is powerful — and dangerous. We learned early that without guardrails, the agent would occasionally make catastrophic decisions:

  • Restarting the only running pod (causing complete outage)
  • Scaling to 50 instances for a minor latency spike (cost explosion)
  • Rolling back twice in a row (data inconsistency risk)

We built 11 configurable safety rules that act as hard constraints on the agent's action space. The key insight: guardrails don't limit learning — they focus it. By preventing obviously bad actions, the agent explores the useful strategy space more efficiently.

4. Memory Makes the Difference

The single most impactful feature was persistent memory. Without it, EVOO is just a fancy chatbot that makes random decisions. With it, EVOO becomes an agent that accumulates wisdom.

Each experience tuple captures the full context:

$$\text{Experience} = \langle s_{\text{incident}},\ m_{\text{before}},\ \pi_{\text{strategy}},\ a_{\text{tools}},\ m_{\text{after}},\ t_{\text{recovery}},\ R,\ \tau \rangle$$

Where $s$ is the incident state, $m$ are metrics, $\pi$ is the strategy, $a$ are the actions taken, $R$ is the reward, and $\tau$ is the timestamp. This tuple is stored and indexed by incident type, enabling the agent to recall: "The last 3 times I saw a memory leak, strategy X scored 92 and strategy Y scored 45."

How We Built It

Architecture Decisions

We built EVOO on the scale-agentex framework, which provides:

  • Temporal for durable workflow orchestration (survives crashes, retries automatically)
  • State machines for managing the learning loop lifecycle
  • ACP (Agent Communication Protocol) for real-time UI communication

The agent is decomposed into specialized roles:

Role Responsibility Implementation
Planner Analyze incident, select strategy LLM reasoning + memory lookup
Executor Run remediation tools with safety checks Temporal activities + guardrails
Evaluator Score the outcome Reward formula + LLM judge
Learner Store experience, update rankings Memory persistence + strategy manager

The Learning Loop

The core of EVOO is a 6-state learning loop that runs for $N$ iterations (default: 10):

IDLE → DETECTING → PLANNING → EXECUTING → EVALUATING → LEARNING → IDLE

Each state is a separate workflow that can fail independently and be retried. The state machine orchestrates transitions and maintains the complete context across the loop.

LLM Integration

All reasoning is powered by LLM calls via OpenRouter (DeepSeek V3). We chose to make every analytical decision LLM-driven rather than rule-based:

  • Log analysis: LLM reads simulated logs and identifies root cause patterns
  • Incident prediction: LLM predicts incident type from raw metrics
  • Strategy planning: LLM considers incident context + historical data to select strategy
  • Outcome evaluation: LLM judges remediation quality as a qualitative assessor

This means EVOO's reasoning improves as LLM capabilities improve — no code changes needed.

Simulated Production System

We built a realistic production simulator that:

  • Generates 6 types of incidents with configurable severity
  • Produces correlated metrics (a CPU spike affects latency, a memory leak affects availability)
  • Responds to remediation actions with probabilistic outcomes
  • Tracks infrastructure cost for each action

The simulator is stateful — actions have lasting effects, and the system can partially recover or degrade further based on what the agent does.

Challenges We Faced

Challenge 1: Making the Agent Actually Learn

The hardest part wasn't building the agent — it was making it demonstrably improve. Early versions showed flat or noisy reward curves because:

  • The reward function was too simple (binary success/failure)
  • The strategy catalog was too small (agent ran out of things to try)
  • The exploration rate didn't decay (agent never settled on good strategies)

Solution: We expanded to 18 strategies across 6 incident types, designed a multi-factor reward function, and implemented $\varepsilon$-decay. The result: a clear upward trend in reward and downward trend in recovery time across 10 cycles.

Challenge 2: Balancing Autonomy and Safety

An agent that can scale your infrastructure and restart your services needs guardrails. But too many guardrails make the agent useless. We iterated through several approaches:

  • v1: No guardrails → agent occasionally caused worse outages
  • v2: Hard-coded limits → too rigid, couldn't adapt to different environments
  • v3: Configurable guardrail engine with 11 rules → right balance

The key insight was making every guardrail threshold configurable via environment variables. Different organizations have different risk tolerances — EVOO adapts to theirs.

Challenge 3: Making Learning Visible

An agent that learns invisibly doesn't build trust. We needed users to see the evolution happening. This led us to build:

  • Sparkline reward charts showing the trend across cycles
  • Phase indicators (🌱→🌿→🌳→🏆) showing the agent's maturity
  • Before/after comparison tables showing early vs. late performance
  • Tool call visibility showing exactly what the agent did and why
  • Guardrail verdicts showing when and why actions were blocked

The UI doesn't just show results — it tells the story of an agent growing from novice to expert.

Challenge 4: LLM Consistency

LLMs are non-deterministic. The same incident can produce different analysis, different strategy selections, and different evaluations. This adds noise to the learning signal.

Solution: We use structured prompts with explicit output formats, combine LLM judgment with deterministic reward formulas, and rely on the law of large numbers — over 10+ cycles, the noise averages out and the signal emerges.

What's Next

EVOO currently learns in simulation. The next frontier is real-world learning:

  1. Shadow mode — EVOO observes real incidents, recommends actions, but doesn't execute. Humans validate. EVOO learns from the gap between its recommendation and the human's action.

  2. Federated learning — Multiple EVOO instances across organizations share anonymized strategy rankings. A new deployment starts with collective wisdom instead of zero knowledge.

  3. Continuous strategy generation — Instead of selecting from a fixed catalog, EVOO uses LLMs to generate novel strategies for incidents it hasn't seen before.

The vision: every production system in the world has an EVOO that never sleeps, never forgets, and never stops improving.

🫒 EVOO — Because your infrastructure deserves an SRE that learns from every incident.

Built With

  • adk
  • agentex
  • deepseek
  • openai
  • rilo
  • temporal
Share this project:

Updates

Submission history