Incident Bridge: Where Infrastructure Telemetry meets Revenue Protection

The Problem

When a critical outage hits, SREs race through server logs with no idea which enterprise clients are affected. Support gets flooded with angry tickets but has no idea why. The result: SLA breaches, panicked account managers, and clients churning before anyone even picks up the phone.

Incident Bridge closes that gap autonomously. The moment an anomaly is detected, a multi-agent AI pipeline cross-references live Dynatrace Grail telemetry - infrastructure error logs enriched with business context events (ARR, SLA thresholds, account ownership) - against your Zendesk support queue — segmenting affected accounts, predicting penalty exposure, and pre-drafting synchronized communications for both teams.

What It Is: Closed-Loop Autonomous Response

Incident Bridge is an AI-orchestrated incident response platform that detects, triages, and coordinates communications during critical outages.

  1. Automated Telemetry Sentry: Instead of waiting for lagging static alerts, Gemini continuously queries live log streams (via Dynatrace Grail) to spot early signs of transactional failure.
  2. Blast Radius & SLA Predictor: The platform executes a two-query DQL architecture. First, it fetches infrastructure logs to identify failing merchant IDs. Second, it pulls live business context (ARR, SLA thresholds, account ownership). It then cross-references this with active Zendesk support tickets to project legally binding penalty liabilities at +15 and +30 minute milestones.
  3. Multi-Agent Coordination: We use specialized Gemini models for deep diagnostic reasoning and brand-safe communication drafting, paired with a deterministic SLA engine that converts live business context into exact penalty liabilities — no LLM arithmetic in the money path.
  4. Human-in-the-Loop Dispatch: SREs review the AI's findings in a clean dashboard. With one click, the platform executes a multi-channel sync to publish updates and notify Account Managers.

The Proactive Risk Matrix

Most incident response tools are just notification engines. Incident Bridge separates judgment from arithmetic: Gemini interprets and diagnoses, while deterministic TypeScript computes the exact set logic and penalty figures — so the dollar amounts on screen are never an LLM's guess.

  • $T$ = Set of Merchant IDs extracted from Grail log errors
  • $C$ = Set of Merchant IDs extracted from Zendesk ticket tags

By comparing these sets, we identify:

  • Escalations ($T \cap C$): Users experiencing errors who have filed a ticket. We auto-link these to a master Problem ticket.
  • Silent Impact ($T \setminus C$): VIP clients failing in telemetry who haven't complained yet. This is our highest-value feature—enabling Support to reach out and offer reassurance before the client even notices the issue.

Core Innovations

  • Visualized AI Reasoning: No black boxes. Our dashboard features a streaming diagnostic terminal so SREs can watch the agent parse logs, query Zendesk, and calculate risk step-by-step.
  • Dynatrace Grail as Business Intel: Most observability tools treat Dynatrace as a log store. Incident Bridge treats Grail as a business intelligence layer by injecting CloudEvents 1.0 bizevents at simulation time, making ARR, SLA contractual thresholds, and account ownership queryable alongside infrastructure metrics.
  • Adaptive Decaying Polling: To avoid API rate limits, our system sweeps for new Zendesk tickets using a decaying scheduler (e.g., at 2, 5, and 10 minutes) rather than continuous polling.
  • Native Structured Outputs: We use the official @google/genai SDK's structured JSON schemas to ensure Gemini's outputs are strictly deterministic, preventing downstream integration crashes.
  • Model Context Protocol (MCP): We use MCP to standardize tool calls, letting our agents seamlessly interact with Dynatrace and Zendesk in real-time.

How We Built It

  • Google Cloud Run: A serverless TypeScript/Node.js agent runtime orchestrating the entire cognitive lifecycle.
  • Gemini Models: Gemini Flash handles rapid log diagnostics and proactive sweeps; Gemini Pro manages complex textual reasoning and communication drafting.
  • Google Cloud Firestore: Provides real-time state synchronization between our agents and the React dashboard.
  • Google Identity Services (GSI): Ensures secure, zero-trust authentication verified by an Express backend.

What's Next

  • Predictive Incident Modeling: Forecasting outages based on historical telemetry trends to pre-stage mitigations.

Built With

Share this project:

Updates