• 

  SentinelAgent live dashboard — real-time provider health monitoring across Claude, OpenAI, MCP Server, and local BiLSTM fallback

• 

  SentinelAgent dashboard showing graceful degradation — Claude and OpenAI both down, local BiLSTM serving responses with zero API dependency

Inspiration

Production LLM agents fail silently when infrastructure breaks. A Claude API brownout, an MCP server crash — users get blank screens. There's no graceful story. I wanted to build one.

What it does

SentinelAgent is a resilient multi-agent system with a 3-tier fallback chain: Claude → OpenAI → Local BiLSTM (4.9M params, 77.31% accuracy, 1.6M tweets)

When Claude fails, it automatically routes to OpenAI. When ALL cloud LLMs fail, my own PyTorch BiLSTM model serves sentiment analysis locally with zero API dependency. A live dashboard shows provider health, latency, error rates, and a real-time request log. Chaos injection buttons let you kill any provider mid-demo to watch the fallback activate instantly.

How we built it

• Flask API with /chat, /status, and /chaos endpoints
• Health monitor with circuit breaker logic per provider
• TrueFoundry AI Gateway for multi-LLM routing
• PyTorch BiLSTM model as offline fallback
• MCP client for tool augmentation (skipped gracefully when down)
• Vanilla JS dashboard with 3-second status polling

Challenges we ran into

• Making BiLSTM load conditionally so the app runs without torch installed
• Circuit breaker recovery logic — knowing when to retry a dead provider
• MCP failing silently vs loudly — had to distinguish connection errors

Accomplishments that we're proud of

• A real trained ML model as fallback, not a mock
• The chaos demo is visually compelling — judges can break it live
• Built in one night from scratch, fully working

What we learned

• How to design agentic systems that degrade gracefully
• Circuit breaker patterns applied to LLM infrastructure
• TrueFoundry AI Gateway routing and observability

What's next for SentinelAgent

• Full TrueFoundry Gateway integration for observability dashboard
• Streaming responses
• Deploy on AWS EC2 with public URL

Built With

• flask
• groq
• llama-3.3-70b
• mcp
• python
• pytorch
• truefoundry-ai-gateway