FYF Studio: Agentic Cinema & Telemetry

An agentic business video studio orchestrating Google ADK agents, 20 automated QA gates, and real-time ClickHouse Cloud telemetry with an MCP Data Officer.

Comment

Inspiration## Inspiration

Making high-quality business explainer videos in emerging markets like Myanmar is expensive and slow. Local production agencies take weeks and charge thousands of dollars. Meanwhile, off-the-shelf generative AI video tools suffer from three fatal flaws for enterprise use:

  1. Uncontrolled Brand Drift: Whenever you ask an AI model to revise a single line of script, it regenerates the entire video, randomly changing approved brand colors, characters, or established visual assets.
  2. Broken Regional Typography: Headless renderers and generic AI tools routinely mangle complex Burmese Unicode diacritics and text layouts.
  3. Zero Financial Transparency: Teams burn through model API quotas with no audit trail of which scene, provider call, or retry consumed their budget.

We built FYF Studio to make generative video deterministic, brand-safe, and accountable. We wanted a studio where approved scenes can be locked against AI hallucinations, and where every token, render latency, and quality check streams directly into ClickHouse Cloud so engineering and business teams have complete visibility into their production pipeline.

What it does

FYF Studio provides an end-to-end video production pipeline with strict brand controls and an embedded telemetry ledger:

  • Brief to Multi-Format Presets: Operators enter a product concept or script draft. The studio instantly configures production parameters across Burmese flagship brand explainers, social ads, and product launches with adaptive 16:9, 1:1, or 9:16 aspect ratios.
  • Brand Lock & Scene Controls: Operators can activate an immutable Story Lock and freeze specific scenes (narration lock, visual lock, timing lock). When collaborating with our AI Creative Director, the system cryptographically protects locked scenes, ensuring subsequent AI revisions never alter approved brand assets or wording.
  • Chat + Canvas Workspace: An interactive storyboard canvas allows granular scene editing, script adjustments, and version rollbacks without touching locked content.
  • Remotion Video Rendering: Turns structured scene data into frame-accurate motion graphics, native Burmese typography, and character mouth movements driven by phonetic viseme cues.
  • 20-Gate Automated QA Inspector: Before a video hits the approved library, an automated inspector runs 20 deterministic gates—verifying peak audio headroom, visual evidence consistency, and phonetic lip-sync alignment.
  • ClickHouse Cloud Telemetry Ledger: Every job automatically streams latency, token usage, model calls, and cost metrics into ClickHouse Cloud via replay-safe tables.
  • Natural Language Data Officer (ClickHouse MCP): Operators can ask questions like "How many video jobs are recorded in total, and what are their statuses?" right in the dashboard. An ADK agent queries ClickHouse Cloud in real time using the official ClickHouse MCP server and returns the verified numbers badged "✓ Answered from a live ClickHouse query".

How we built it

We designed an architecture that strictly separates creative orchestration from brand safety, video rendering, and telemetry persistence:

  • Agentic Orchestration: Google ADK (google-adk) powers the production agent, using Gemini 2.5 and 3.7 Flash on Google Cloud Vertex AI for script synthesis and visual evidence planning.
  • Brand Lock Engine (backend/projects/scene_locks.py): We implemented a deterministic lock store that validates every edit command against active scopes. If an AI proposal touches a locked scene or brand boundary, the engine safely rejects the change (lock_conflict), preserving brand integrity.
  • ClickHouse Storage Engine: Telemetry runs on ClickHouse Cloud hosted on GCP (asia-southeast1). We designed four core tables using ReplacingMergeTree engines (video_pipeline_jobs, video_qa_records, video_scene_telemetry, and video_vertex_calls) to ensure data ingestion is idempotent and replay-safe.
  • Durable Outbox Pattern: To prevent dropping metrics during network blips or burst loads, the backend appends events locally to a JSONL outbox before a background worker drains them into ClickHouse Cloud.
  • Official ClickHouse MCP Integration: For the Data Officer, we launched @modelcontextprotocol/server-clickhouse as a stdio server wired into Google ADK's MCPToolset. The agent dynamically inspects database schemas, runs analytical SQL queries against ClickHouse Cloud, and returns grounded answers.
  • Frontend & Rendering: A Next.js App Router frontend with Tailwind CSS connects to a FastAPI backend. Final video rendering uses Remotion running on headless Chromium, packaged in a Docker container and deployed to Google Cloud Run.

Challenges we ran into

  • Preventing AI Drift with Granular Locks: Getting an LLM agent to obey immutable scene boundaries through system prompts alone failed during edge cases. We solved this by enforcing explicit backend pre-command validation hooks: if a command payload targets a locked scene ID, the backend rejects it with a structured lock_conflict error before any state changes occur.
  • The 18-Second Query Viewport Shift: In our initial demo walkthrough, executing a full 100-row Creation Timeline query took 18 seconds and inserted a tall table that pushed the Data Officer input completely off-screen. We resolved this by adding compact analytical summary presets (Cost Intelligence) that execute in 47ms, and implementing smooth auto-centering on the Data Officer card once query results stabilize.
  • Burmese Unicode in Headless Chromium: Headless browsers often fail to render complex Burmese vowel clusters and diacritics properly without desktop OS font packages. We had to bundle custom Burmese Unicode fonts and build visual snapshot tests inside Remotion to guarantee clean typography.
  • Audio and Frame Drift: Aligning Gemini-TTS audio clips with Remotion animation frames required exact audio duration probing via ffprobe before compiling the final render manifest, ensuring that character lip-sync visemes and background music align to the exact millisecond.

Accomplishments that we're proud of

  • Building a genuine human-in-the-loop studio where brand locks protect approved work while the AI freely iterates on unlocked scenes.
  • Shipped genuine ClickHouse MCP integration where an ADK agent autonomously queries ClickHouse Cloud to explain its own production metrics.
  • Sub-150ms analytical query response times on ClickHouse Cloud presets (Cost Intelligence runs in just 47ms).
  • Shipped two complete end-to-end video productions passing all 20 automated QA gates deployed on Google Cloud Run.
  • Kept average production cost under $0.01 per 30-second explainer video.

What we learned

  • Prompts are not guardrails: For enterprise brand safety, business rules and locked assets must be enforced by deterministic backend validation logic, not prompt instructions.
  • Telemetry should be decoupled: Using an outbox pattern for telemetry prevents third-party analytics delays from impacting core user latency. ClickHouse Cloud's ReplacingMergeTree table engine proved ideal for handling replay-safe telemetry without duplicate rows.
  • MCP is a game-changer for agentic analytics: Giving the Data Officer access to official ClickHouse MCP tools transformed it from a standard chatbot into a genuine analytics partner that inspects tables and provides verified metrics.

What's next for FYF Studio: Agentic Cinema & Telemetry

  • Multi-Agent Specialist Teams: Introducing specialized agent sub-roles (sound designer, compliance reviewer, colorist) that collaborate on the living canvas.
  • Automated A/B Variant Testing: Generating multiple hook variations for social ads and streaming viewer retention metrics back into ClickHouse to automatically discover high-converting formats.
  • Multi-Dialect Expansion: Extending native voice, typography, and cultural nuance support across other regional Southeast Asian languages and dialects. And moe video feature and skills v2.

Built With

Share this project:

Updates

Submission history