AI Cinematographer — Pre-Production Engine

An intelligent agentic pre-production studio that transforms scene descriptions into technically precise scene descriptions and photorealistic storyboard images using Google cloud and grafana

Comment

Inspiration

Photography is one of my hobby and often I find myself fascinated by the scene and mood constructions of a movie director. I wanted to create something in that space. After some research, I understood that the pre-production phase involves a lot of iteration between the director and the cinematographer to finally land on the scene. This project aims to make that iteration more efficient using genAI.

What it does

It takes a simple narrative scene description and acts as a fully autonomous pre-production studio by doing three things:

  1. Director of Photography (DoP) Reasoning It transforms your narrative concept into a structured, highly precise set of technical cinematography specifications. (For example: "Medium Close-Up (MCU) / 85mm Focal Length / Harsh overhead fluorescent lighting, strong shadows under eyes, slightly underexposed to enhance grittiness, high contrast.")

  2. Storyboard Visualization It passes those exact technical specifications to a specialized rendering agent that generates a stunning, photorealistic storyboard image depicting exactly how the scene will look through the lens of the chosen camera setup.

  3. Autonomous Budget & Observability Auditing Behind the scenes, it utilizes an autonomous "Studio Executive" agent (powered by the Model Context Protocol) to query Grafana Prometheus metrics in real-time. This agent intercepts the OpenTelemetry token metrics emitted during the generation process and instantly produces a markdown budget report, while Grafana Agent Observability tracks the safety, pipeline DAG, and evaluator fulfillment (e.g. Prompt Injection, Cinematic Feasibility) of the entire process.

How we built it

The application is orchestrated via a Python backend and Streamlit UI, divided into four major components:

  1. Director of Photography (DoP) Reasoning Agent We used gemini-2.5-flash heavily guided by a System Prompt and a structured Pydantic output schema. This forces the LLM to translate narrative scene descriptions into highly specific, professional cinematography data contracts (focal lengths, camera angles, lighting ratios, and composition instructions).

  2. Storyboard Image Rendering Agent We leveraged gemini-2.5-flash-image to instantly visualize the shot. It takes the highly technical output generated by the DoP agent and renders a photorealistic storyboard image that adheres exactly to the requested focal lengths and lighting.

  3. Full Grafana Observability Pipeline (agento11y + MCP) We implemented a robust production observability layer:

Agent Observability (agento11y) — Tracks multi-agent interactions, model IDs, latency, and DAG dependency relationships between the DoP reasoner and image renderer. We also implemented Custom AI Evaluators (e.g., prompt_injection_detection, cinematic_feasibility, hallucination_detector) to automatically grade generation quality and safety. OpenTelemetry Traces & Metrics — Distributed span hierarchies and gen_ai.client.* metrics are natively exported to Grafana Cloud via OTLP. Model Context Protocol (MCP) Budget Agent — We integrated the Grafana MCP Server to power a "Studio Executive" agent. This agent dynamically executes PromQL queries against Grafana to fetch OpenTelemetry token metrics, calculating live cost estimations for the scene.

  1. Streamlit Web UI We built an interactive presentation layer in Streamlit that offers a side-by-side visualization of the generated storyboards, the technical shot specifications, and the executive budget reports.

Challenges we ran into

  1. Imagen vs. Gemini Native Image Gen — Had to pivot from Google's Imagen API to Gemini's native image generation (gemini-2.5-flash-image) mid-build, which required completely different response parsing logic.
  2. Dual Authentication Paths — Switching from a simple API key to Vertex AI's Application Default Credentials (ADC) for Cloud Run deployment required building a config toggle and debugging Docker container auth.
  3. Agento11y SDK Trial-and-Error — The Grafana observability SDK is new with limited docs; we had to reverse-engineer the correct API for system prompts (GenerationStart(system_prompt=...)), token reporting (usage=TokenUsage(...)), and agent DAG linking (GenerationStart(id=...)).
  4. Silent Fallbacks Masking Bugs — Our try/except ImportError with dummy classes silently swallowed real errors, causing cryptic crashes like 'GenerationStart' has no attribute 'mode' far downstream.
  5. Cloud Run Deploy Lifecycle — Learned that pushing to GitHub doesn't auto-update Cloud Run — each code change requires a fresh gcloud run deploy --source . to rebuild and redeploy the container.
  6. 3 hours prior to deadline when I was about to submit, I checked the full rules and realized I do not have the mcp integration which is a requirement for grafana track and only have observability, after a few more iterations, the updated app finished with 30 minutes left on the clock.

Accomplishments that we're proud of

  1. Building the app and deploying it in google cloud run
  2. Building the observability system using grafana cloud (I used langfuse and opik before, learned grafana for this project).
  3. Building the full project in a very short time, utilizing antigravity, gemini spark and chats, grafana ai skill that made it easier to implement the observability

What we learned

  1. Learned how to take an idea into a real product and deploy it on google cloud
  2. Learned a little more about google cloud and how to navigate it (still feel lost to be honest)
  3. Learned how to deploy using google cloud run and push changes using gcloud cli
  4. Learned the agent observability tech stack of Grafana (did not know it existed before this hackathon)
  5. Created a few evaluation strategies to catch hallucination, prompt injection attack etc. ## What's next for AI Cinematographer — Pre-Production Engine
  6. Phase 2 features: i. multi-turn chat to iterate on a generated image (tweak the camera angle, change lighting condition etc) ii. use veo models to maybe generate a short video snippet instead of image as an additional option. iii. Can it become the director and direct a full scene given a storyline?
  7. Phase 2 observability: The current observability and evaluators barely scratch the surface, we need to have end to end observability and CI/CD implemented. Need prompt versioning, human-in-the-loop to align the judge llm.
  8. phase 3 features: i. add tool call and browsing capabilities, so, the agent can do some research before generating the scene ii. create multiple generations from one question to give different perspectives

Built With

  • agento11y
  • google-cloud
  • google-genai
  • grafana
  • opentelemetry-api
  • opentelemetry-exporter-otlp
  • opentelemetry-sdk
  • pillow
  • pydantic
  • pytest
  • python-dotenv
  • streamlit
  • vertex-ai
Share this project:

Updates

Submission history