Aegis-Agent of Agents

An AI Agent That Supervises AI Agents

Comment

Inspiration

Teams are shipping fleets of AI agents into production — but nobody supervises the supervisors. When an agent silently leaks PII, burns through its token budget, or starts failing tool calls, nobody notices until it's expensive or dangerous. I wanted to build the missing layer: an agent whose entire job is watching other agents, catching failures in real time, and handling the full incident-response loop autonomously — with the governance controls an enterprise would actually require.

What it does

Aegis is an autonomous supervisor agent that watches a fleet of production AI agents and runs a complete incident-response loop on every failure:

Detect → Diagnose → Decide → Govern → Remediate → Report

  • Detect: A detector service classifies incoming fleet events (PII leaks, tool failures, budget overruns, low-confidence outputs) and files incidents in Firestore — the live dashboard flips red.
  • Diagnose: Gemini 3.5 Flash Lite (via Vertex AI) performs structured root-cause diagnosis with severity scoring.
  • Decide: A deterministic policy engine maps diagnosis → action, so critical decisions aren't left to LLM vibes.
  • Govern: High-impact actions pass a human-in-the-loop approval gate (toggleable via AEGIS_AUTO_APPROVE), and every step lands in an append-only Firestore audit trail.
  • Remediate: Aegis quarantines the offending agent and the fleet goes green again.
  • Report: It writes a postmortem, and a reporting chatbot answers live questions about incident metrics straight from Firestore.

How we built it

  • Google ADK is the brain: an LlmAgent with typed tools that, for each open incident, calls get_open_incidents → diagnose → decide_action → remediate → write_postmortem — a real tool-call loop, not a prompt chain.
  • Gemini 3.5 Flash Lite via Vertex AI powers structured root-cause diagnosis and the dashboard reporting chatbot.
  • Cloud Run hosts all three services: detector, supervisor (ADK web UI + agent API), and dashboard.
  • Firestore holds incident state, the quarantine list, and the append-only audit log.
  • Pub/Sub carries fleet events with a dead-letter topic for poison messages; Secret Manager holds credentials.
  • The whole loop is asynchronous and event-driven — no human clicks required between failure and postmortem.
  • 58/58 unit tests passing (pytest tests/ -v), repo public with an architecture diagram and reproducible setup instructions.

Challenges we ran into

  • Auth reality check: the original plan used a legacy Gemini API key, but legacy AIzaSy… keys could no longer be generated on this account. The fix was a better architecture anyway: pointing the diagnoser at Vertex AI with service-account auth — which also made the "real Google Cloud proof" story stronger.
  • Keeping the agent honest: it's tempting to let the LLM decide everything. I learned to keep the LLM for judgment (diagnosis) and deterministic code for authority (policy, quarantine, approvals) — that's what makes the governance gate trustworthy.
  • End-to-end timing: a real detect→remediate loop involves Pub/Sub latency, cold starts, and LLM calls, so choreographing a demo that fits a tight video window took careful seeding and rehearsal.

Accomplishments that we're proud of

  • A genuinely autonomous loop running on real Google Cloud infrastructure — live Vertex AI calls returning 200 in Cloud Run logs, with real Gemini-generated root causes.
  • The governance story: a one-env-var human-in-the-loop gate plus an append-only audit trail, visible in the product, not just in the README.
  • It's not a chatbot in a nice UI — the supervisor reasons, plans, and invokes tools asynchronously, with Firestore as long-term memory.
  • Shipped solo, end-to-end: three Cloud Run services, event pipeline, dashboard, reporting agent, tests, and demo.

What we learned

  • How Google ADK's typed tool-calling turns an LLM into a reliable operator — and where to draw the line between model judgment and deterministic policy.
  • Practical Vertex AI + service-account auth from Cloud Run (and the failure modes of API-key auth).
  • That observability and auditability aren't features you bolt on — for agents that act on other agents, they're the product.

What's next for Aegis — Agent of Agents

  • Multi-fleet federation: one Aegis supervising several fleets across projects/regions, with aggregated risk scoring.
  • Richer remediation playbooks: rollback, traffic shifting, and budget throttling — beyond quarantine.
  • Deeper Gemini Enterprise integration: agent discovery and registry so new fleet agents are auto-enrolled for supervision.
  • Policy-as-code UI: let teams write governance rules without touching env vars.
  • Anomaly detection: proactive signals (latency drift, cost anomalies) before hard failures occur.

Built With

Share this project:

Updates

Submission history