CineAgent Studio

Script. Storyboard. Sound. Directed by AI.

Comment

CineAgent Studio 🎬🤖

Autonomous Multi-Agent AI Film Crew powered by Gemini Enterprise, Google Cloud Log Analytics, and ClickHouse Vector Engine.

💡 Inspiration

Traditional film pre-production is a fragmented, multi-month marathon. Directors, screenwriters, production designers, storyboard artists, audio supervisors, and financial analysts work in isolated silos. When an indie filmmaker or studio executive has a visionary concept, iterating on early drafts requires coordinating dozens of creative departments, costing tens of thousands of dollars and weeks of turnaround time.

With recent breakthroughs in Gemini 2.5 multimodal models, ClickHouse high-performance vector databases, and Model Context Protocol (MCP), we asked ourselves:

What if an entire Oscar-caliber Hollywood film crew could collaborate autonomously in real-time, translating a single logline into a production-ready film bible, scene-by-scene screenplay, visual storyboards, synthesized voice acting, audio soundscapes, and market analytics within seconds?

This vision drove the creation of CineAgent Studio—an agentic virtual backlot that democratizes cinematic storytelling and brings studio-grade pre-production to every creator.

🎥 What It Does

CineAgent Studio acts as an autonomous virtual studio lot where specialized AI agents collaborate across the entire pre-production lifecycle:

  1. Executive Producer & Director Agent: Deconstructs raw premises or uploaded screenplays into a complete Film Bible—including tone guidelines, target demographics, character psychological profiles with costume notes, and dynamic 3-to-6-act dramatic structures.
  2. Screenwriter Agent: Adapts the concept into industry-standard formatted screenplay scenes with scene sluglines, vivid action blocks, character dialogue, and emotional subtext.
  3. Storyboard Director Agent: Plans cinematography and camera angles (wide anamorphic, over-the-shoulder, Dutch angles) and generates 16:9 cinematic shot concepts.
  4. Production Designer Agent: Designs set architectures, atmospheric lighting, and hero props equipped with "Anti-Amnesia Consistency Shields" to prevent continuity errors across scenes.
  5. Audio Department Agent: Crafts orchestral score themes, foley sound effects, and dramatic auditory cues synchronized to the scene's emotional pacing.
  6. Actor Voice Vault & Speech Synthesizer: Assigns distinct voice profiles using Google Cloud Text-to-Speech (Journey & Neural2 voices), enabling instant vocal performance playback.
  7. Market Analyst & Continuity Supervisor Agent: Computes box-office benchmarks, dramatic tension curves, dialogue density ratios, and verifies narrative continuity by querying ClickHouse via MCP vector search.
  8. Interactive Director's Cut: Allows creators to provide granular director's notes on individual scenes and trigger real-time, context-aware rewrites.

🛠️ How We Built It

CineAgent Studio is architected around a distributed, multi-agent pipeline with asynchronous streaming, vector indexing, and enterprise observability.

+-----------------------------------------------------------------------------------+
|                                 USER / DIRECTOR                                   |
+-----------------------------------------------------------------------------------+
                                          │
                                          ▼
+-----------------------------------------------------------------------------------+
|                        CineAgent Studio Web UI (FastAPI)                          |
+-----------------------------------------------------------------------------------+
                                          │
                     [ Stage 1: Executive Producer (Gemini 2.5) ]
                                          │
                     [ Stage 2: Screenwriter Agent (Gemini 2.5) ]
                                          │
             ┌────────────────────────────┼────────────────────────────┐
             ▼ (Parallel Stage 3)         ▼ (Parallel Stage 3)         ▼ (Parallel Stage 3)
+────────────────────────+   +────────────────────────+   +────────────────────────+
|  Storyboard Director   |   |  Production Designer   |   |    Audio Department    |
| (Cinematography/Shots) |   | (Sets, Props, Costume) |   | (Score, Foley, Voices) |
+────────────────────────+   +────────────────────────+   +────────────────────────+
                                          │
                                          ▼
+-----------------------------------------------------------------------------------+
|                      ClickHouse Vector & Asset Vault                             |
|  - Table: `projects` (Production metadata & status)                               |
|  - Table: `scenes` (768-dim Vector Embeddings for Semantic Search)               |
|  - Table: `dialogues` (Dialogue lines, character tags & emotion)                  |
|  - Table: `storyboards` (Visual camera shots & lighting prompts)                  |
|  - Table: `production_design` (Set architecture, hero props & costumes)           |
|  - Table: `audio_post` (Soundtracks, foley effects & audio cues)                  |
|  - Table: `generated_images` (Persisted AI concept frames; excludes SVGs)         |
|  - Table: `actor_voice_vault` (Synthetic & real voice profiles & accents)        |
|  - Table: `dialogue_audio` (Generated audio takes & voice line history)           |
+-----------------------------------------------------------------------------------+
                                          │
                                          ▼
+-----------------------------------------------------------------------------------+
|                 Google Cloud Logging & Log Analytics (BigQuery)                   |
|  - Non-blocking Background Transport shipping structured JSON telemetry           |
|  - Model Thinking Telemetry (`thoughts_token_count`, prompt/candidate tokens)     |
+-----------------------------------------------------------------------------------+

1. Foundation Models & Multimodal Generation

  • Gemini 2.5 Flash (gemini-2.5-flash): Powered by the official @google/genai Vertex AI SDK, handling screenplay generation, scene breakdowns, and structured JSON output with strict safety guardrails.
  • Text Embedding (text-embedding-004): Generates 768-dimensional dense vector embeddings $\vec{e} \in \mathbb{R}^{768}$ for all screenplay scenes.
  • Google Cloud Text-to-Speech: Synthesizes high-fidelity vocal takes across diverse gender, tone, and accent profiles (Journey-F, Journey-D, Neural2).

2. ClickHouse Cloud & MCP Vector Engine

We deployed an 8-table normalized schema on ClickHouse Cloud to serve as the unified persistent memory and analytics engine:

  • Scenes are embedded and indexed using cosine distance. For query vector $\vec{q}$ and stored scene vector $\vec{v}$: $$\text{Cosine Similarity}(\vec{q}, \vec{v}) = \frac{\vec{q} \cdot \vec{v}}{|\vec{q}|2 |\vec{v}|_2} = \frac{\sum{i=1}^{768} q_i v_i}{\sqrt{\sum_{i=1}^{768} q_i^2} \sqrt{\sum_{i=1}^{768} v_i^2}}$$
  • The Continuity Agent connects to ClickHouse via the ClickHouse MCP Server (mcp-clickhouse) to query semantic scene parallels in sub-15ms.

3. Pipeline Latency Optimization

To achieve near real-time rendering, downstream creative departments run concurrently via asyncio.as_completed() and stream results to the client over Server-Sent Events (SSE).

Compared to sequential execution ($T_{\text{seq}}$), the parallel architecture achieves a significant latency reduction ($\Delta T$): $$T_{\text{seq}} = T_{\text{EP}} + T_{\text{Screenwriter}} + \sum_{k \in \mathcal{D}} T_k$$ $$T_{\text{parallel}} = T_{\text{EP}} + T_{\text{Screenwriter}} + \max_{k \in \mathcal{D}}(T_k) + \epsilon$$ $$\Delta T = 1 - \frac{T_{\text{parallel}}}{T_{\text{seq}}} \approx 54.2\%$$ where $\mathcal{D} = {\text{Storyboard}, \text{ProductionDesign}, \text{Audio}, \text{Market}}$ and $\epsilon$ is network overhead.

4. Production Observability & Telemetry

  • Dual-dispatch logging emits structured JSON to stdout and streams asynchronously to Google Cloud Logging & BigQuery Log Analytics.
  • Tracks Model Thinking Telemetry (thoughts_token_count, prompt_token_count, candidates_token_count) and correlates multi-agent pipelines with X-Request-ID.

🚧 Challenges We Faced

  1. Eliminating "Narrative Amnesia" Across Agents:
    Early tests showed downstream agents inventing contradictory props or wardrobe changes across scenes. We solved this by creating an Anti-Amnesia Consistency Shield: the Production Designer locks set architecture and hero props into ClickHouse vector memory, which subsequent agents query before writing new scenes.

  2. Progressive JSON Parsing over SSE:
    Streaming multi-agent output progressively over Server-Sent Events while maintaining valid JSON schemas required designing an event-driven stream handler that emits discrete typed events (bible_ready, scenes_ready, department_complete) rather than raw token chunks.

  3. MCP Integration in Cloud Environments:
    Ensuring the ClickHouse Model Context Protocol server operated seamlessly across local macOS development, Docker containers, and serverless Google Cloud Run required building an adaptive MCP client that handles authentication and fallback to native ClickHouse drivers when needed.

  4. Calibrating Dramatic Tension Math:
    Building a meaningful dramatic pacing curve required normalizing tension scores $\tau_i \in [1.0, 10.0]$ against pacing tags ($\text{SETUP} \to \text{SUSPENSE} \to \text{CLIMAX} \to \text{RESOLVE}$), computing a continuity index $\mathcal{C}$: $$\mathcal{C} = \frac{1}{N-1} \sum_{i=1}^{N-1} \left( \text{sim}(\vec{e}i, \vec{e}{i+1}) \times \left(1 - \frac{|\tau_{i+1} - \tau_i|}{10}\right) \right)$$

📚 What We Learned

  • Specialized Multi-Agent Crews Outperform Monolithic Prompts: Breaking filmmaking down into distinct role-based personas (Director, Screenwriter, Sound Designer) produced far richer creative nuance and realistic screenplay format than a single mega-prompt.
  • Vector Search as Narrative Memory: Storing scene embeddings in ClickHouse transformed how agents reference earlier acts, enabling instantaneous continuity validation across complex multi-act stories.
  • Enterprise Observability is Essential for Agentic Workflows: Having real-time visibility into token usage, latency per agent, and Gemini thinking token telemetry in Google Cloud Log Analytics made debugging and optimizing prompt latency painless.

🚀 What's Next for CineAgent Studio

  • Veo 2 Video Generation Integration: Direct integration with Google's Veo video generation model to turn storyboard camera prompts into high-definition animated video scenes.
  • Dynamic Multi-Track Audio Mixing: Automated mixing of Cloud TTS character dialogue takes with ambient foley audio and orchestral soundtrack cues.
  • Full Screenplay PDF Export (Final Draft & Fountain): One-click export to industry-standard .fdx and .pdf formats for instant Hollywood submission.

Built With

Share this project:

Updates

Submission history