Director agentic AI

Inspiration

On any real film set, a director is constantly watching the frame — catching a shaky shot, a bad angle, or flat lighting before it ruins a take. Most indie creators and solo filmmakers don't have that second pair of trained eyes. We wanted to build an AI agent that acts like a real director on set — watching every frame in real time and giving direct, actionable feedback, the same way a human director would call out "step back a touch" or "give the lens more room."

What it does

Director Agentic AI (Slate) watches a live camera feed and continuously analyzes three things a real director cares about — Framing, Motion, and Light. It scores each in real time and gives spoken-style director's notes ("pull the energy back", "settle down, slower") until the shot holds. Once a shot meets the bar, it's captured as a "Take" with its framing/motion/light scores and the number of notes it took to get there — building a live record of every take, just like a director's shot log.

How we built it

We built Slate as a web app powered by Gemini for real-time multimodal shot analysis and Google Cloud Agent Builder to orchestrate the agent's decision loop — evaluating framing, motion, and light frame-by-frame and generating natural-language director's notes. We integrated Parallel to ground the agent's feedback with real cinematography and composition knowledge pulled from the web, so the notes it gives reflect actual filmmaking best practices rather than generic guesses. The frontend captures the live camera feed and streams frames to the agent, which returns live notes and scores rendered directly in the "Director's notes" and "Takes" panel.

Challenges we ran into

Getting the agent to give feedback that felt natural — like a real director, not a generic AI — took a lot of prompt iteration. Coordinating Parallel's research calls with Gemini's real-time analysis without slowing down the feedback loop was also tricky, since the agent needed to stay responsive while still pulling in grounded knowledge.

Accomplishments that we're proud of

Getting a genuinely live, working agent loop — not a mock or scripted demo — where framing/motion/light scores and notes update in real time based on the actual camera feed, backed by real research via Parallel.

What we learned

We learned a lot about orchestrating multi-step agent reasoning with Gemini and Agent Builder, how to use Parallel to ground an agent's output in real-world knowledge, and how to translate raw model output into feedback that reads like something a human director would actually say.

What's next for Director agentic AI

Next, we want to add support for multi-camera setups, let the agent remember a director's personal style/preferences across sessions, and expand scoring to include composition rules like rule-of-thirds and color grading feedback.

Built With

• claude
• css
• html
• javascript
• ovencv
• python