RelayGrid Cinema

A shared human-agent control room for 220 live media streams — Gemini reasons over the grid through real MCP tool calls; every fix still needs an explicit human Approve & Execute click.

Comment

Inspiration

Streaming/CDN infrastructure already does some self-healing -- AWS Elemental MediaLive can auto-failover on audio loss, multi-CDN setups auto-reroute around a degraded network. But that automation is rigid and per-stream: a fixed threshold on one signal, no view of the fleet as a whole, and nothing an operator can question or reason with. An operator watching hundreds of streams still has to notice a cross-stream pattern by hand -- correlating bitrate, audio, and subtitle telemetry -- before any of that per-stream automation even applies.

Broader IT/network operations have already started moving past this: modern AIOps platforms correlate across systems and gate risky actions behind human approval instead of running everything autonomously. That same reasoning-plus-approval layer doesn't yet exist for media/streaming ops specifically -- the tools above still stop at a single fixed rule per stream.

RelayGrid Cinema brings that layer to media ops. The agent understands the whole grid, investigates a fault in plain language across bitrate, audio, and subtitle signals together, and turns raw telemetry into an explained, ready-to-approve fix -- instead of a fixed if-audio-silent-then-failover rule with no fleet-wide context. Every mutating action still stays behind an explicit human "Approve & Execute" click: the same trust-tiered autonomy AIOps already validates for risk-gating automated fixes, applied here to a domain that doesn't have it yet.

The underlying pattern -- a live grid governed through an MCP server -- is one we'd explored before, in an earlier personal hackathon project: a single grid, one scripted incident scenario, no persistent automation. RelayGrid Cinema is a new codebase, written from scratch for this contest and this domain: a standing policy engine with its own AUTO/APPROVAL tiers and live rule authoring, a natural-language reporting layer with drill-down, and real running integrations with both Grafana Cloud and ClickHouse, all on top of Google Cloud Agent Builder / Vertex AI. Same instinct for the interaction model; everything built here is new, and built for media/streaming operations specifically.

What it does

The left-hand grid shows live stream health (CDN, bitrate, FPS, audio, subtitle sync, status) across 220 streams. Ask Gemini in plain language -- "show streams with bitrate below 3 Mbps or audio/subtitle issues in the last 24h" -- and it filters the grid live via a real MCP tool call, explains why a specific stream matched, and prepares a remediation action (failover CDN + restart audio encoder) as a non-mutating preview. Nothing changes until a human clicks Approve & Execute on the action card.

A standing policy engine evaluates the whole grid continuously: AUTO-tier rules (e.g. "audio desync on an otherwise healthy stream -> restart the encoder") resolve on their own and get logged; APPROVAL-tier rules (e.g. "bitrate critically low -> switch CDN") escalate a pending action card and wait for a human click. New rules can be authored live, in plain English, from the Policy Rules panel. A natural-language Reports panel answers fleet-wide questions with real aggregates and click-to-drill-down into the underlying records. A one-click "Run full scenario" button drives the whole guided flow automatically for live demos, stopping right before the human-confirmation step.

Why WebMCP/MCP is essential

Every grid mutation -- filter, explain, preview, execute, undo -- goes through one shared set of MCP tools, described once and consumed by two callers: native WebMCP (document.modelContext.registerTool, for any MCP-aware agent browser) and our own Gemini-powered chat panel. The execute_action tool is deliberately not exposed to Gemini's function-calling toolset at all -- it's only reachable from the UI button -- so the mutating step can never be taken by the model itself, only by a human who saw the preview.

How we built it

A domain-agnostic condition-tree query engine and batch-action planner sits underneath everything, generic over any record type. Media & Streaming is the first concrete domain built on that contract -- the same engine is designed to support future domains (e.g. a healthcare worklist) without changes elsewhere. Gemini access goes through a swappable backend interface: both a direct Gemini API backend and a Google Cloud Agent Builder / Vertex AI backend share one function-calling loop and differ only in how they authenticate, so the UI and tool layer never need to know which is active.

Policy-rule authoring is natural language; policy-rule execution deliberately isn't. A plain-English request gets resolved by Gemini's function call into a fixed, typed condition -- every future match against that rule runs through the same deterministic evaluator the rest of the engine already uses, with no further model call. A rule the model could silently reinterpret differently between runs isn't something you can safely wire to real infrastructure, so once a rule exists it behaves exactly like hand-written code -- reproducible and auditable, not "elastic."

Real, live partner integrations run behind one shared seam: Grafana Cloud (the declared Partner Track) via the official mcp-grafana MCP server for real tool-calling, plus every action pushed live into a real Loki stream -- and ClickHouse, tested the same way, with both running together in production. The live app authenticates to Google Cloud via Workload Identity Federation (no service-account key ever created or stored), confirmed live against the public URL itself.

Challenges we ran into

The GCP project enforces iam.disableServiceAccountKeyCreation, which blocks creating a service-account key outright -- the standard way most tutorials wire up Vertex AI from a serverless function. The fix was Workload Identity Federation: exchanging Vercel's own OIDC token for short-lived Google credentials at request time, so no key ever exists on disk or in an env var. Grafana Cloud's own hosted MCP endpoint only supports interactive OAuth today, so real headless tool-calling from a serverless function meant vendoring the official mcp-grafana Linux binary directly into the deployment instead. ClickHouse's official MCP server is a Python process with no Node/Vercel-compatible runtime, so its tool-calling is exercised locally while its direct write path stays live in production.

Accomplishments that we're proud of

  • A shared set of MCP tools used identically by native WebMCP and an in-app Gemini chat loop.
  • Structural safety: the execute step is unreachable by the model, only by a human click.
  • A standing policy engine with AUTO vs. APPROVAL tiers, live rule authoring, and natural-language reporting with real drill-down.
  • Two real, live partner integrations (Grafana Cloud + ClickHouse) behind one shared seam, running together in production.
  • Domain-agnostic query/action engine, proven with one domain and designed for a second.
  • A one-click, repeatable, judge-facing demo path with live progress tracking.
  • A distinctive "master control room" UI, responsive from phone through desktop.

What we learned

That the safest place to put a trust boundary is structural, not promised -- keeping execute_action out of the model's own toolset entirely turned "the model shouldn't act without approval" from a prompt instruction into something actually true regardless of what the model outputs. Also that "natural language in, deterministic code out" is a much safer pattern for anything wired to real infrastructure than letting the model re-interpret a rule on every match.

What's next for RelayGrid Cinema

Implement the second domain (Healthcare/Radiology worklist) this engine was designed to support. Widen policy-rule authoring's grammar from a single flat condition to the same compound AND/OR/NOT trees already supported for hand-authored rules, so an operator can register something like "be more sensitive to quality drops on premium Sunday broadcasts than on night reruns" directly from chat. Separately, staged/timed remediation ("try a lighter fix first; escalate only if it hasn't helped within N seconds") would need real per-record state tracking across ticks, which the current stateless-per-tick evaluator doesn't have yet.

Built With

Share this project:

Updates

Submission history