Sage The MemoryAgent

Introduction

Every on-call engineer has done this: something breaks at 2 AM, you fix it, and then three weeks later the exact same thing breaks again because nobody wrote it down. The knowledge lives in Slack threads that scroll away, in one person's head, in tribal lore that new hires spend months absorbing.

Cloud agents have the same problem on fast-forward. You point an LLM at your infrastructure, it tries something, it fails, and the next session starts from zero. Nothing was learned. The same $5 mistake gets repeated forever because there's no mechanism to turn corrections into durable, applied memory.

We wanted to build the agent that actually learns — not just stores a log, but retrieves and acts on past corrections before it makes tool calls. That's the whole idea behind Sage.

What Sage is

Sage is a self-correcting AI agent for cloud operations. It has three modes that all share the same memory and learning loop:

Mode	What it does
Simulated / Offline	Heuristic model against an in-memory sandbox. Nothing leaves your machine.
Simulated / Qwen	Qwen LLM reasons against a simulated Alibaba Cloud MCP server. Real model, no real resources.
Live Cloud	Qwen LLM against the real Alibaba Cloud MCP server. Actual inventory and deployment operations.

The key: the same code path runs in all three. The memory loop doesn't care whether the substrate is simulated or real.

How the learning loop works

Here's the core idea, step by step:

1. You give Sage a cloud task — say, "deploy an ECS instance in us-east-1."
2. It tries. It fails — maybe it forgot to pass a VSwitchId (us-east-1 requires one).
3. You submit a correction: here's what went wrong, here's the fix.
4. Sage stores it as a rule with a confidence score, embeds it for semantic search.
5. Next time a similar task comes in, Sage retrieves that rule before it acts and blocks the failing action.

The payoff is something you can demo live: run a task, watch it fail, teach it, run the same task again, and watch it succeed — not because you changed the prompt, but because Sage remembered.

The memory system

Sage doesn't just keep one big vector store. It maintains separate memory types:

• Rules — operational corrections with confidence and utility scores. You can pin the ones you trust and retire the ones that stop being useful.
• Skills — ordered tool-call sequences that worked. When a similar task shows up, Sage reuses the recipe.
• Episodes & Cases — records of past runs, so Sage can say "last time we tried this, it failed because…"
• Provenance — every rule traces back to the specific correction and run that created it.

Rules have a lifecycle. Each one tracks a confidence \(c \in [0, 1]\) and a utility \(u\) that decays over time:

$$c_{t+1} = \alpha \cdot c_t + (1 - \alpha) \cdot \mathbb{1}[\text{correct}]$$

Low-utility rules get consolidated or retired automatically. The store doesn't bloat.

How we built it

• Backend: Python + FastAPI. One api.py handles everything — session-scoped agents, the admin-token gate, credential routes, memory CRUD.
• Frontend: React + TypeScript + MUI. We added a centralized error interceptor that shows judge-friendly messages instead of raw header names.
• LLM: Qwen via DashScope. The caller is OpenAI-compatible, so you could swap in another model with a config change.
• Embeddings: An independent embedding provider backs semantic rule retrieval — you can paraphrase a correction however you want and still find it.
• Cloud integration: The Alibaba Cloud MCP server (44 tools). Every mutating operation is wrapped in a narrowly-typed, fail-closed caller. The model never gets raw API access.
• Deployment: Running on an Alibaba Cloud ECS instance in us-east-1, fronted by HTTPS (ngrok static domain) so Web Crypto works in the browser.

Challenges we ran into

Web Crypto on plain HTTP. Deployed over http://, the browser silently drops crypto.randomUUID() — no secure context, no Web Crypto. Our session IDs and idempotency keys broke in ways that only showed up in the deployed UI, never in local dev. We fixed it with a typed-array RFC-4122 fallback in the client, and put HTTPS in front.

Two credential paths, one wrong landmine. The web app reads Alibaba Cloud credentials from the browser session, not from env vars. If you accidentally conflate the two, a judge could end up routed to the mainland region (which requires passport verification). We defaulted the credentials dialog to us-east-1 and added explicit error copy.

Keeping the memory loop honest across modes. The correction → retrieval → constrained-action loop had to work identically whether the backend was simulated or hitting real cloud. We discovered the tool-restriction contract was documented but not actually threaded through the Run and AgentLoop classes. That was the load-bearing fix — without it, the live demo would exercise a different code path than the safe one.

Fail-closed means fail-closed. With no admin token, the API had to return 503 — not silently serve an unprotected dashboard. Unit tests said it worked. The deployed box disagreed. We caught it by actually testing against the live deployment, not just the test suite.

What we learned

• Applied memory beats stored memory. An agent that logs corrections but doesn't retrieve them before acting hasn't actually learned anything. Retrieval-before-action is the whole game.
• Fail-closed design has to be verified in production. The failure mode on the deployed box ("page loads, every action 503s") looked nothing like the unit test failure.
• Secure context is a hard requirement, not a nice-to-have. The moment you deploy outside localhost, you need HTTPS or half your browser APIs silently vanish.

What's next

• Persist memory across server restarts (right now it's in-memory per session).
• Expand the read-only inventory surface so more operations are safe by default.
• Add a confidence-weighted auto-retirement policy so the maintenance pass self-tunes the rule store over time.

Importance

Most cloud agents are either safe but forgetful (sandbox-only) or capable but reckless (full API access, no memory). Sage is a small attempt at the thing that's actually useful: an agent that accrues operational knowledge from real mistakes and spends it against real infrastructure — safely, with guardrails at three layers (admin token gate, server-side mutation switch, tool allowlist).

Built With

• codex
• matt-pocock-skills
• mcp
• python
• qwen
• workflows