AegisOps Autopilot

Qwen-powered production incident autopilot with memory, agent review, tool-backed evidence, and human approval gates.

Comment

Judge-first summary

AegisOps Autopilot is a Qwen Cloud production-incident autopilot safety harness, not a passive incident dashboard. Qwen plans through five scoped tools, the same tool surface is exposed through OpenAPI and MCP for judge verification, and a human approval gate blocks risky production mutations before they execute.

The headline ablation is full workflow 0.988 versus single-agent baseline 0.420, a +0.568 absolute gain from memory, tool-backed evidence, policy checks, dry-run remediation, and approval gating. The expanded stress benchmark (report) covers 14 production-style incident scenarios, 14 services, 70 approved-path tool calls, and 14/14 blocked-mutation checks. The adversarial authority benchmark (report) adds 56 corrupted model/tool-boundary attacks with 56/56 authority-boundary checks passed, 14/14 active-incident scoping checks, 14/14 approval-bypass blocks, 14/14 unknown-tool rejections, and 14/14 policy hard-stop checks. A verified one-shot live Qwen smoke proof (report) confirms qwen-cloud / qwen-plus, five Qwen tool schemas, 1518 ms latency, and redacted credential state without saving a secret. The project is intentionally adversarial: make Qwen useful enough to remediate real incidents, but constrained enough that it cannot silently ship unsafe changes.

Separate Alibaba Cloud backend proof recording: https://youtu.be/KECJK5LgGOA

Live Alibaba ECS proof endpoints:

Inspiration

Production incident response is still stuck between read-only chatbots and unsafe full automation. A serious Autopilot Agent should do the repetitive evidence gathering and planning work, but it should also preserve traceability, policy checks, and human approval for risky changes.

What it does

AegisOps turns ambiguous production alerts into a traced remediation workflow. It recalls relevant incident memory, gathers log, metric, change, policy, and dry-run evidence, asks Qwen Cloud to diagnose and plan, convenes specialized agent roles, proposes reversible remediation, requires human approval for risky actions, and stores post-incident lessons.

The demo covers reliability latency, privacy and PII risk, and billing-risk incidents. The impact target is practical production operations: faster triage, better evidence completeness, and fewer unsafe autonomous actions.

How we built it

The app uses a TypeScript Node API, a React dashboard, and a Qwen Cloud client compatible with OpenAI-style chat completions. The backend includes persistent incident memory, deterministic judging fixtures, a five-tool incident registry, a Qwen Function Calling loop, risk-scored remediation planning, OpenAPI tool endpoints, a lightweight MCP stdio server, and an Alibaba Cloud proof endpoint.

The five incident tools are log_search, metric_probe, change_graph, policy_check, and remediation_simulator. The same registry backs the live Qwen Function Calling loop, the HTTP/OpenAPI tool endpoints, and the MCP stdio server.

Qwen Cloud usage

src/server/agent/qwenClient.ts calls Qwen Cloud through the DashScope OpenAI-compatible API. A one-shot live Qwen smoke proof is verified in reports/live_qwen_smoke_proof.md with qwen-cloud, qwen-plus, five tool schemas, latency, and redacted credential state. The public Alibaba ECS deployment exposes Qwen Cloud provider metadata at /api/health, including model, base URL, timestamp, and redacted credential state. Public reviewer mode can run deterministic fixtures so judges can test the same workflow without exposing or burning a private key; setting DASHSCOPE_API_KEY or QWEN_API_KEY flips the same backend into live Qwen Cloud mode.

When Qwen returns tool_calls, AegisOps executes only incident-scoped tools, overrides any model-supplied incident ID with the active workflow incident, appends role=tool evidence, and asks Qwen for the final diagnosis. The adversarial authority benchmark proves that corrupted Qwen self-reports, cross-incident tool arguments, unknown tool names, and policy hard-stop attacks cannot cross that deterministic authority boundary.

Custom tool, OpenAPI, and MCP integration

OpenAPI spec: agents/aegisops/openapi.yaml

Capability manifest: agents/aegisops/cap-manifest.json

MCP stdio server: src/server/mcp/aegisopsMcp.ts, runnable with pnpm run mcp:stdio.

The dashboard includes a Judge rubric evidence panel that maps the official 30/30/25/15 criteria to concrete repository evidence, so judges can quickly verify Qwen integration depth, engineering controls, impact, and documentation readiness from the running demo.

Alibaba Cloud deployment proof

Qwen Cloud Base URL code proof: https://github.com/Oxygen56/aegisops-autopilot/blob/main/src/server/agent/qwenClient.ts

Alibaba proof endpoint code: https://github.com/Oxygen56/aegisops-autopilot/blob/main/src/server/cloud/alibabaProof.ts

Separate backend proof recording: https://youtu.be/KECJK5LgGOA

Live demo: http://101.201.33.56/

Live health endpoint with Qwen Cloud metadata: http://101.201.33.56/api/health

Live proof endpoint: http://101.201.33.56/api/alibaba/proof

Verified deployment report: reports/alibaba_deployment_proof.md

Verification command: pnpm run deploy:verify -- http://101.201.33.56

Challenges we ran into

The hardest part was balancing autonomy with safety. The project uses approval gates, deterministic fallback fixtures, scoped tool execution, explicit evidence boundaries, and adversarial authority tests so the agent can be useful without hiding risk. Another challenge was making the same tool surface useful for Qwen, OpenAPI, MCP, the dashboard, and a judge-facing demo path.

Accomplishments that we're proud of

AegisOps is not a dashboard-only demo. The same incident tool registry is exposed to Qwen, OpenAPI, and MCP. The UI includes a judge evidence panel mapped to the hackathon rubric, and the repository includes tests, evaluation reports, ablation evidence, a 14-scenario stress benchmark, a 56-scenario adversarial authority benchmark, architecture assets, build provenance, Alibaba Cloud deployment proof, and a reproducible judge packet.

What we learned

Production-grade agents need evidence boundaries as much as reasoning quality. Qwen Cloud is strongest when it can call scoped tools, compare evidence, and explain why a remediation should or should not proceed while deterministic code keeps tool authority, incident scope, and mutation approval outside the model.

What's next for AegisOps Autopilot

Next steps are deeper observability integrations, more incident classes, stronger cost controls, and team workflow features for incident commanders and service owners.

Built With

Share this project:

Updates