AEGIS

The autonomous guardian that stops costly AI agent loops before they drain your budget

Comment

About the Project

Inspiration

The inspiration for AEGIS came from a painful reality we witnessed while building AI agents for production: agents get stuck in expensive loops.

During development of e-commerce AI assistants, we noticed a recurring pattern when an agent encountered ambiguous queries or missing data, it would repeatedly call the same tool 10, 20, sometimes 50+ times, burning through thousands of tokens and dollars before anyone noticed. Traditional observability tools showed us the problem after the damage was done. We asked: What if an agent could watch other agents and heal them in real-time?

This led to AEGIS, an autonomous guardian that doesn't just monitor AI agents, but actively intervenes to prevent costly failures before they impact your bottom line.

Accomplishments That I Proud Of

  • First-of-its-Kind Meta-Agent System: I built an AI agent that watches and heals other AI agents—a pattern we believe will become standard in production AI infrastructure.
  • Production-Grade Security Architecture: Despite the hackathon timeline, we implemented enterprise-level security practices: zero-trust frontend, environment-based secrets, CORS lockdown, and masked API responses.
  • The Config Playground "Wow Moment": I are incredibly proud of the split-screen interface where developers can watch an agent's system prompt rewrite itself in real-time. This visual demonstration instantly communicates the product's value and makes the abstract concept of "auto-healing" tangible and understandable.
  • Hackathon Compliance Without Compromise: I successfully navigated the Arize track's strict requirement for code-owned runtimes while still leveraging Google Cloud Agent Builder's power. This required deep understanding of both platforms and creative architectural decisions.
  • Real-Time WebSocket Integration: Building a truly real-time dashboard that updates live as traces are generated and heals are applied was technically challenging but resulted in a dramatically better user experience.
  • LLM-as-a-Judge Implementation: Using Gemini to evaluate trace quality and determine when intervention is needed was a sophisticated approach that elevated the system beyond simple rule-based detection.
  • Premium UI/UX: Creating a dashboard with smooth animations, thoughtful micro-interactions, and a cohesive design system, while also building complex backend logic.

What I Learned

Building AEGIS taught us profound lessons about AI agent observability and self-healing systems:

  1. The Meta-Agent Pattern: I learned that the most effective way to manage AI agents is with another AI agent. By giving our Sentinel agent access to Arize Phoenix traces via MCP, it could "introspect" its own operational data and make intelligent decisions about when to intervene.
  2. Real-Time Intervention vs. Post-Mortem Analysis: Traditional APM tools are reactive. I discovered that by streaming traces through WebSockets and using LLM-as-a-judge to evaluate loop severity in milliseconds, I could achieve zero-downtime healing, fixing agents while they're still running.
  3. The Psychology of Trust: One of my biggest insights was that developers won't adopt auto-healing systems unless they can see what's happening. This drove our decision to build a split-screen "Config Playground" where users watch the system prompt rewrite itself in real-time, building trust through transparency.
  4. OpenInference as the Universal Language: I learned that standardized tracing formats (OpenTelemetry/OpenInference) are critical for building portable, vendor-agnostic observability layers.

How I Built It

AEGIS Sentinel follows a strict three-tier architecture designed for production-grade security and scalability:

1. The Frontend (Next.js 14 + TypeScript)

We built a premium SaaS dashboard. The UI features:

  • Live Traces Feed: Real-time scrolling visualization of agent spans, color-coded by health status
  • Config Playground: A split-screen interface where developers chat with an agent on the left while watching its system prompt auto-rewrite on the right
  • Agents Fleet Management: Monitor multiple agents, pause/resume them, and view health scores
  • Settings Control Center: Configure loop detection thresholds, token budgets, and notification preferences

All frontend code uses a zero-trust API model; it never directly calls Arize or Google Cloud. Instead, it communicates exclusively with our Python backend via secure REST endpoints and WebSockets.

2. The Backend (Python FastAPI)

The backend serves as the orchestration layer:

  • Vertex AI SDK Integration: We use google-cloud-aiplatform to invoke Google Cloud Agent Builder agents programmatically (not just the visual console), ensuring we can wrap calls in OpenInference for tracing
  • Arize Phoenix MCP Client: Our backend connects to the Phoenix MCP server to query traces, spans, and evaluations at runtime
  • WebSocket Server: Pushes real-time updates to the frontend whenever a loop is detected or a heal is applied
  • Background Sentinel Worker: An asyncio task that continuously polls Arize for anomalous trace patterns

3. The Sentinel Logic (The "Brain")

The core innovation is our autonomous healing loop:

  1. Detect: The MCP worker queries recent traces and identifies patterns like "same tool called > 5 times in 10 seconds"
  2. Judge: I use Gemini as an LLM-as-a-judge to evaluate the severity of the loop and generate a recommendation
  3. Heal: The system uses the Vertex AI SDK to dynamically update the target agent's system instructions (e.g., adding "STOP after 3 failed attempts")
  4. Notify: A WebSocket event is pushed to the frontend, showing the developer exactly what changed and why

4. Security & Production Readiness

  • All API keys stored in .env files, never exposed to the frontend
  • CORS locked to specific frontend origins
  • Masked API keys returned to UI (e.g., arize_****1234)
  • Structured for deployment on Google Cloud Run (backend) and Vercel (frontend)

Challenges I Faced

Challenge 1: Arize Track Compliance

Problem: The hackathon rules stated: "The visual Agent Builder alone is not supported for tracing integration. You must be able to instrument your code directly."

Solution: I initially tried using the drag-and-drop Agent Builder console, but quickly realized this would disqualify us from the Arize track. We pivoted to using the Vertex AI SDK (vertexai.generative_models) to invoke agents programmatically, allowing us to wrap every call with openinference-instrumentation-vertexai. This was a critical architectural decision that saved our submission.

Challenge 2: Real-Time Trace Analysis Without Overhead

Problem: Querying Arize Phoenix on every user request added unacceptable latency.

Solution: I implemented a background polling worker using asyncio that runs independently of the main request cycle. It checks for anomalies every 5 seconds and caches results, ensuring the frontend gets real-time updates without slowing down agent responses.

Challenge 3: Safe Auto-Healing Without Breaking Agents

Problem: Automatically rewriting system prompts is risky—what if the "fix" makes the agent worse?

Solution: I added a human-in-the-loop approval mode (configurable in Settings). By default, AEGIS shows the proposed fix in the Config Playground and waits for developer approval. For advanced users, there's an "Auto-Apply" toggle that lets the Sentinel heal autonomously. We also implemented rollback logic—if the healed agent performs worse on the next query, the system reverts to the previous prompt.

Challenge 4: MCP Integration Complexity

Problem: The Arize Phoenix MCP server documentation was sparse, and we struggled to understand how to query traces programmatically.

Solution: I dug into the Phoenix source code and discovered we could use the Phoenix REST API directly as a fallback. We built a hybrid approach: try MCP first, fall back to REST if needed. This made our integration more robust and gave us deeper insights into how MCP servers work under the hood.

Challenge 5: Being a Solo Developer Against the Clock

Problem: Building a full-stack meta-agent system with real-time observability, self-healing, and premium UI is typically a team effort. As a solo developer, context switching between GCP IAM issues, OTLP endpoint debugging, Pydantic validation errors, and Framer Motion animations was exhausting. There were moments I genuinely wished I had started two weeks earlier or had a teammate to handle the frontend while I wrestled with backend infrastructure.

Solution: I leaned heavily on AI-assisted development for boilerplate and debugging, but maintained strict manual review of all architecture-critical code. I also adopted a "fail fast, fix faster" mentality—when the REST API auth kept failing, I pivoted to MCP within hours instead of spending days debugging a dead end. The constraint of being solo actually forced cleaner abstractions; if I couldn't explain a module simply to myself at 3 AM, it needed refactoring.

Built With

Languages & Frameworks:

  • Python 3.11+
  • TypeScript
  • Next.js 14 (App Router)
  • FastAPI
  • React 18

AI/ML & Agent Infrastructure:

  • Google Cloud Vertex AI Agent Builder SDK
  • Google Gemini (gemini-2.0-flash)
  • OpenInference (openinference-instrumentation-vertexai)
  • Arize Phoenix MCP Server (@arizeai/phoenix-mcp)

Frontend Technologies:

  • Tailwind CSS
  • shadcn/ui
  • Framer Motion
  • Lucide React (icons)
  • Zustand (state management)

Backend Technologies:

  • Uvicorn (ASGI server)
  • WebSockets (real-time updates)
  • Pydantic (data validation)
  • python-dotenv (environment management)
  • asyncio (background workers)

Cloud & DevOps:

  • Google Cloud Platform (GCP)
  • Google Cloud Run (backend deployment)
  • Vercel (frontend deployment)
  • Arize Phoenix Cloud (observability SaaS)

Design & UX:

  • Glassmorphism effects
  • Bento grid layouts
  • Real-time data visualization

Security:

  • Zero-trust API architecture
  • CORS lockdown
  • Environment-based secrets management
  • Masked API key responses

Hackathon Partner Integrations:

  • Arize: Phoenix MCP server for trace introspection, loop detection, and auto-healing
  • Google Cloud: Vertex AI Agent Builder SDK for code-owned agent runtime

What's Next for AEGIS

Short-Term

  • Multi-Cloud Agent Support: Extend AEGIS to monitor agents built on OpenAI, Anthropic, and open-source frameworks like LangChain and LlamaIndex, not just Google Cloud Agent Builder.
  • Advanced Healing Strategies: Implement more sophisticated auto-heal techniques beyond system prompt rewriting, including dynamic tool parameter adjustment, automatic fallback to alternative tools, and context window optimization.
  • Team Collaboration Features: Add multi-user support with role-based access control, audit logs, and team-wide notifications when agents are healed using Google Cloud IAM.
  • Cost Analytics Dashboard: Build detailed cost breakdowns showing exactly how much money AEGIS has saved, with projections and ROI calculations.

AEGIS Sentinel started as a solution to a painful problem we faced, but we believe it points to the future of AI infrastructure: agents that manage themselves. We're excited to continue building toward that vision.

Built With

  • arize-phoenix-cloud
  • arize-phoenix-mcp
  • asyncio
  • fastapi
  • framer-motion
  • gemini
  • google-cloud
  • google-cloud-agent-builder
  • google-cloud-run
  • google-gemini
  • javascript
  • lucide-react
  • next.js-14
  • openinference
  • pydantic
  • python
  • python-dotenv
  • react-18
  • rest-api
  • shadcn/ui
  • tailwind-css
  • typescript
  • uvicorn
  • vercel
  • vertex-ai-sdk
  • websockets
  • zustand
Share this project:

Updates