Inspiration
In high-throughput microservice architectures (such as payments, e-commerce, and logistics engines), sudden database connection drops or downstream API latency spikes usually lead to catastrophic failure chains. Traditional circuit breakers simply throw a 503 Service Unavailable error, leading to lost transactions and degraded user experience. Inspired by high-reliability transaction processors like Razorpay's Vulcan architecture, we wanted to build a resilient, agentic safety net using Gemini and Google Cloud that transparently heals backend failures without dropping state or crashing end-user flows.
What it does
Vulcan-Shield is an autonomous AI agent layer that acts as a dynamic circuit breaker and dynamic fallback environment for mission-critical microservices:
- Real-Time Fault Interception: Continuously monitors serverless/microservice execution streams and intercepts 5xx infrastructure errors or network timeouts before they hit the client.
- Dynamic Mock Environment Synthesis: Using Gemini 1.5, the agent analyzes the incoming payload, failed API endpoint schema, and historical system context to instantaneously synthesize a zero-state, compliant mock environment, returning a valid provisional payload to keep the user flow alive.
- Autonomous Queueing & State Replay: The agent isolates mutated backend events inside a secure staging queue (Google Cloud Pub/Sub) and actively polls down-stream health nodes. Once the target microservice recovers, the agent autonomously replays queued mutations, reconciling state asynchronously.
How we built it
- Intelligence Layer: Built with Gemini 1.5 via the Google GenAI SDK, leveraging Structured Outputs and Function Calling to inspect payload structures and generate accurate mock API fallbacks.
- Orchestration & Workflow: Developed using Google Cloud Workflows and Python fast-api microservices to intercept service layer exceptions.
- Data & Queueing: Powered by Google Cloud Pub/Sub for event-driven message queuing and transaction tracking.
- Host Infrastructure: Deployed on Google Cloud Run to handle high-throughput, low-latency agentic execution.
Challenges we ran into
- Sub-Second Latency Requirements: Standard LLM calls can be too slow for transaction microservices. We optimized Gemini's system instructions and used structured JSON schemas to ensure fallback synthesis completes within milliseconds.
- Ensuring Idempotency on Replay: Preventing duplicate database mutations when replaying queued transactions after downstream nodes healed required strict transaction signature generation.
Accomplishments that we're proud of
- Built a zero-downtime microservice safety network that gracefully handles downstream database crashes.
- Successfully executed end-to-end self-healing cycles without losing a single transaction state during simulated load tests.
- Designed a fully decoupled architecture adhering to Google Cloud best practices.
What we learned
- How to harness Gemini's structured function calling for non-chat, high-utility background execution tasks.
- Strategies for designing robust, agentic fallback loops that prevent cascading microservice outages in enterprise applications.
What's next for Vulcan-Shield
- Multi-Cloud Node Health Prediction: Integrating predictive latency telemetry to reroute transactions before failures happen.
- Automated Root-Cause Analysis (RCA): Enabling the agent to generate detailed post-mortem reports and GitHub pull requests to patch code bugs automatically.
Built With
- fastapi
- gemini-api
- google-cloud-pubsub
- google-cloud-run
- google-cloud-workflows
- python