StreamOps AI: Autonomous Media Incident Orchestrator

An autonomous Gemini agent that connects to ClickHouse to monitor live streaming telemetry, detect network anomalies, and auto-remediate CDN degradations for flawless global broadcasts.

Comment

Inspiration

During high-concurrency live global broadcasts—such as championship sporting events, season finales, and digital concerts—every second of stream degradation translates into thousands of lost viewers and forfeited advertising revenue. Modern streaming architectures generate terabytes of real-time Quality of Experience (QoE) telemetry across CDNs, edge nodes, and viewer devices. Traditional Site Reliability Engineering (SRE) and media operations teams rely on fragmented dashboards and manual SQL queries, taking 15 to 30 minutes to detect and remediate CDN edge failures. We built StreamOps AI to transform reactive incident response into an autonomous, closed-loop media operations workflow powered by Google Cloud Agent Platform and ClickHouse.

What it does

StreamOps AI is an autonomous media operations agent designed to monitor global broadcast streams, identify localized CDN degradations in real time, and execute multi-step failover remediations:

  • Sub-Second Telemetry Analytics: Continuously queries billions of telemetry events via ClickHouse to aggregate rebuffering ratios, dropped frame rates, ad pod render failures, and active session distribution.
  • Deterministic Anomaly Detection: Identifies abnormal regional QoE degradation (e.g., buffering exceeding 4% or ad-insertion failure spikes) across multi-CDN delivery networks.
  • Autonomous Remediation: Executes dynamic traffic shifts by adjusting CDN edge weights and routing policies, rerouting live viewer traffic to optimal fallback providers without service disruption.
  • Live Incident Post-Mortems: Generates structured incident briefs and telemetry breakdowns for media production engineers.

How we built it

  • Agent Intelligence & Orchestration: Built using Google Cloud Agent Platform (powered by Gemini) to handle multi-step deterministic reasoning and dynamic tool calling.
  • High-Throughput Telemetry Store: ClickHouse Cloud acts as the real-time analytical database, ingesting streaming metrics and running aggregation queries over massive session logs.
  • Tool Gateway & Integration Layer: Developed a FastAPI orchestration backend hosted on Replit, integrating with ClickHouse and exposing specialized tools for anomaly detection, regional QoE analysis, and CDN routing policy updates.
  • Agent Protocols: Connected the orchestration gateway directly to Gemini via Model Context Protocol (MCP) and OpenAPI specs to ensure zero-hallucination, data-grounded actions.
  • Operations Dashboard: A clean, responsive dark-mode mission control UI providing real-time telemetry visibility, live concurrent viewer stats, and an interactive agent dispatch terminal.

Challenges we ran into

  • Protocol Alignment & Tool Schemas: Ensuring seamless schema mapping between the Agent Platform's MCP execution layer and the FastAPI backend required precise OpenAPI contract definitions to prevent execution dropouts.
  • Simulating High-Volume Edge Telemetry: Constructing a realistic multi-region streaming telemetry dataset that accurately simulated CDN edge failures, ad-pod drops, and viewer migration patterns.
  • Zero-Hallucination Guardrails: Tuning system prompts to ensure the agent strictly bases remediation actions on deterministic ClickHouse query results rather than speculative assumptions.

Accomplishments that we're proud of

  • Reducing mean time to detect (MTTD) and mean time to remediate (MTTR) streaming bottlenecks from over 15 minutes to under 3 seconds.
  • Delivering a fully functional, closed-loop autonomous system where Gemini actively queries analytical data, analyzes trade-offs between providers, and executes network actions.
  • Creating an end-to-end media engineering pipeline that bridges real-time database infrastructure with agentic AI orchestration.

What we learned

  • The power of combining high-speed analytical engines like ClickHouse with Gemini’s tool-calling capabilities to make instant decisions over massive event streams.
  • How to properly architect agent tools to support deterministic multi-step workflows in enterprise media environments.

What's next for StreamOps AI

  • Predictive Pre-Warming: Utilizing historical broadcast telemetry to predict CDN edge congestion before peak viewership surges.
  • Direct SSAI (Server-Side Ad Insertion) Healing: Expanding automated remediation directly into ad-decision servers to maximize ad delivery completion rates.
  • Multi-Cloud Edge Routing: Integrating direct API failovers across AWS CloudFront, Fastly, Akamai, and Cloudflare Media Services.

Built With

Share this project:

Updates

Submission history