-
-
proof of deployment using alibaba
Inspiration
Small engineering teams rarely have 24/7 SRE coverage, but production incidents don't wait for business hours. When an alert fires at 2am, someone has to dig through logs, metrics, dashboards, and runbooks under pressure — and mistakes made in that state are expensive. We also noticed that most "AI SRE" demos stop at chat: you paste in a log and the model summarizes it. That's not incident response, that's search. We wanted to build something that actually investigates — and that never takes an action a human hasn't signed off on.
What it does
OpsPilot is an evidence-first SRE autopilot. Given an incident (seeded demo scenario or a real alert), it:
- Triages the incident with Qwen, classifying severity and type as strict JSON
- Selects tools from a backend-enforced allowlist — the model proposes, the backend validates
- Gathers evidence from logs, metrics, health checks, deployment history, and runbooks
- Diagnoses the root cause, cross-referencing similar past incidents from its own memory
- Recommends remediation — but is never allowed to execute it directly
- Applies a backend risk policy that classifies the action as safe / medium / dangerous
- Pauses for human approval on anything risky, with the reasoning, expected impact, and rollback plan shown up front
- Executes the approved action (simulated), saves what worked as reusable incident memory, and generates a final report
Every one of those steps is written to an auditable timeline — nothing happens off the record.
How i built it
The backend is FastAPI with a thin routing layer over a service layer, backed by SQLAlchemy/SQLite. The agent loop (IncidentAgent) drives a fixed pipeline of Qwen calls, each constrained by a strict JSON schema and validated with Pydantic before the backend acts on it — the model never gets to freelance. A tool registry enforces an allowlist so the LLM can only ever invoke pre-approved investigation tools. A separate policy engine (not the LLM) decides whether an action requires human approval, and an approval workflow blocks execution until a human responds. Every agent step, tool call, approval decision, and audit event is persisted, which powers both the incident timeline UI and a deterministic evaluation runner that replays seeded scenarios for PASS/FAIL scoring.
Qwen is called through Alibaba Cloud's Model Studio (DashScope) OpenAI-compatible API. The whole stack is containerized with Docker Compose and deployed on an Alibaba Cloud ECS instance, with Caddy handling automatic HTTPS via Let's Encrypt in front of the FastAPI backend.
Challenges I ran into
Keeping the LLM "on rails" was the core design problem — it's easy to build a demo where the model just calls whatever tool it wants with whatever arguments it wants, but that's not something you'd trust near production infrastructure. We solved this with a layered-trust model: the LLM proposes, the backend validates tool names against an allowlist, a separate deterministic policy layer decides risk and approval requirements, and dangerous actions are hard-gated behind a human decision — the model has no path to bypass that gate. We also hit the usual "the model didn't return valid JSON" problem, which we handle with strict schema validation and safe fallback decisions rather than letting a bad response crash the run or silently do something risky.
Accomplishments that I am proud of
- A full evidence → diagnosis → policy → approval → remediation loop that's auditable end to end, not just a chat wrapper
- A backend-enforced safety boundary that's structurally impossible for the model to bypass, not just prompt-level guidance
- A deterministic eval suite that proves the same safety behavior holds across multiple incident types
- A live, HTTPS-secured deployment on Alibaba Cloud ECS that judges can actually click into and try
What I learned
Trustworthy AI automation isn't about a smarter model — it's about where you put the boundaries. The most important design decisions in this project were about what the LLM was not allowed to do unilaterally.
What's next for OpsPilot
- Real (non-simulated) remediation integrations behind the same approval gate
- Adaptive investigation — letting the agent request another evidence pass when diagnosis confidence is low, instead of a strictly linear pipeline
- Verifying remediation actually fixed the issue before marking an incident resolved, and feeding failed fixes back into incident memory
Log in or sign up for Devpost to join the conversation.