The problem

When the first dailies review delivery fails, a production coordinator has little time to decide whether the cause is media integrity, delivery infrastructure, or an isolated workflow failure. Raw dashboards contain the clues, but translating them into a reliable and communicable plan is slow—and a guessed diagnosis can make a stressful call time worse.

What CineOps does

CineOps presents one focused incident: DS-042 — dailies review delivery failed. The operator seeds a fully synthetic failure for a recording-safe demo, sees its metric and checksum-mismatch log in Grafana Cloud, and starts an investigation.

The Gemini agent runs through Google ADK on Vertex AI. Before it can create a report, CineOps requires it to call the official grafana/mcp-grafana server over Streamable HTTP. The MCP sidecar is restricted to read/query tools. CineOps records the ordered tool calls and returned evidence, then displays a structured recovery draft with:

  • likely cause and confidence;
  • cited Grafana evidence;
  • estimated time to recover;
  • prioritized recovery steps; and
  • a conditional, crew-facing status update.

The operator can approve the draft under a display name. That approval is persisted in Firestore, but version one has no retry, media, notification, or other external-action adapter. The final decision stays human.

Why Grafana is essential

Grafana is not decoration or an offline export. CineOps sends fresh synthetic OpenTelemetry metrics, logs, and traces to Grafana Cloud. At investigation time, Gemini obtains observability evidence through the official Grafana MCP sidecar. If Grafana/MCP is unavailable or returns no evidence, CineOps explicitly reports that state and withholds a diagnosis instead of fabricating one.

How it is built

The public app is one multi-container Google Cloud Run service:

Browser -> FastAPI operator dashboard -> Firestore
                    |             \
                    |              -> OpenTelemetry -> Grafana Cloud
                    v
            Gemini 2.5 Flash on Vertex AI via Google ADK
                    |
      official grafana/mcp-grafana Streamable HTTP sidecar (read-only)
                    |
             Grafana Cloud metrics, logs, and traces

Secrets are stored in Google Secret Manager and injected only into the container that needs them. The browser never receives Grafana or cloud credentials, and the public API does not accept arbitrary prompts.

Data sources and privacy

Every incident, identifier, log, metric, trace, and piece of UI copy in this project is synthetic and newly created for the demonstration. CineOps retains no footage, cast or crew identity, call-sheet data, or third-party media.

What we learned

An incident agent earns trust by being able to show its work and by stopping when it cannot. Requiring the live Grafana MCP read before a conclusion makes the recovery draft inspectable. Keeping the last action with a named operator makes the demo useful for a real production environment rather than merely impressive in a controlled scenario.

Runtime compliance notes

  • Google Cloud AI: Gemini 2.5 Flash through Vertex AI and Google ADK. No OpenAI, Anthropic, AWS, Microsoft, or other AI model/framework is used.
  • Grafana Labs: Grafana is used at runtime through the official grafana/mcp-grafana server. OpenTelemetry ingestion supplements, but does not replace, that MCP interaction.
  • Public source: the repository contains setup instructions, Cloud Run configuration, tests, and an MIT license. Publish it before final submission.
  • Demo video: show the working public app, Grafana evidence, MCP trail, Gemini draft, and human approval boundary in English within three minutes.

Built With

  • fastapi
  • gemini
  • google-adk
  • google-cloud-firestore
  • google-cloud-run
  • google-secret-manager
  • grafana-cloud
  • grafana-mcp
  • opentelemetry
  • python
  • vertex-ai
Share this project:

Updates

Submission history