Director Desk

Director Desk is an AI Showrunner Studio bringing professional filmmaking workflows to generative AI, from scriptwriting and storyboarding to production, post-production, localization and release.

Comment

Director Desk

Inspiration

AI can now generate stunning individual video clips, but filmmaking is much more than generating isolated scenes. A short film demands consistent characters, believable environments, coherent storytelling, synchronized dialogue, post-production, and a polished release package. Today's generative models excel at creating moments, but they struggle to manage the continuity and production workflow that transforms those moments into a complete film.

At the same time, creators are forced to jump between multiple disconnected tools for scripting, storyboarding, video generation, editing, subtitling, dubbing, accessibility, poster design, and trailer creation. The creative process becomes fragmented long before the film is finished.

We built Director Desk to become the missing layer between creators and AI models—an AI Showrunner Studio that orchestrates the complete filmmaking pipeline. Instead of asking creators to manage dozens of separate production steps, Director Desk acts like a creative director by orchestrating the capabilities of Qwen Cloud into a complete filmmaking workflow.

What it does

Director Desk is an AI Showrunner Studio that transforms a single creative idea into a production-ready short film. Instead of stitching together multiple AI tools and editing software, creators work inside one unified production studio that manages the entire filmmaking lifecycle—from pre-production to release.

Built on Qwen Cloud through Alibaba Cloud DashScope, Director Desk orchestrates Qwen language models, Wan image/video generation models, HappyHorse image-to-video, and Qwen3-TTS into a unified AI filmmaking workflow.

Starting with a single prompt such as "A cyberpunk detective chases a hacker through neon-lit streets," Director Desk orchestrates every stage of production:

  1. Scriptwriting & Storyboarding – Generates an industry-style screenplay, then converts it into a shot-by-shot storyboard with camera directions, lighting, and cinematic composition.
  2. Casting & Asset Production – Creates consistent character portraits, environment references, and voice profiles that serve as the visual and narrative foundation of the film.
  3. Scene Production – Generates individual scenes using Alibaba Cloud DashScope's Wan2.6/2.7 and HappyHorse-1.0 models while preserving continuity through reference-based scene generation and last-frame scene transitions.
  4. Studio Editor – Combines generated scenes into a complete film through a dedicated timeline editor, allowing creators to trim, reorder, and assemble their final cut.
  5. Post Production Studio – Automatically generates subtitles, multilingual dubbing, and accessibility-focused Audio Description (AD) tracks. Smart Speaker Casting keeps dubbed voices consistent across speakers, while subtitle editing remains fully interactive.
  6. Release Studio – Produces theatrical posters, promotional banners, and trailers, allowing creators to prepare their films for distribution directly from the platform.

Rather than generating isolated AI clips, Director Desk manages the complete filmmaking workflow—bringing together creative planning, production, post-production, localization, accessibility, and release into a single cinematic workspace.

How we built it

Architecture

Director Desk is built as a modular AI Showrunner Studio that separates filmmaking into distinct production stages: Pre-Production, Production, Post-Production, and Release. A FastAPI backend orchestrates specialized AI agents while a React + Vite frontend provides a cinematic editing environment for creators.

Technology Stack

  • Frontend: React, Vite, and Tailwind CSS powering a premium studio interface with storyboard management, timeline editing, asset libraries, and real-time production dashboards.
  • Backend: FastAPI (Python) coordinating a multi-agent workflow for screenplay generation, storyboarding, production planning, scene rendering, localization, and release preparation.
  • Infrastructure: PostgreSQL with SQLAlchemy for project metadata, Redis for background jobs, Docker for deployment, and Server-Sent Events (SSE) for real-time production progress.

AI Production Pipeline

Director Desk is powered by Qwen Cloud through Alibaba Cloud DashScope, orchestrating multiple specialized models rather than relying on a single prompt-to-video workflow.

  • Qwen-Plus generates screenplays, storyboards, scene breakdowns, production plans, translations, and creative reasoning.
  • Wan2.6 Text-to-Image generates consistent character and environment references.
  • Wan2.7 Text-to-Video and HappyHorse Image-to-Video render cinematic scenes.
  • Qwen3-TTS powers character voice generation during production.
  • faster-whisper, Edge-TTS, and FFmpeg power subtitle generation, multilingual dubbing, accessibility tracks, and final media processing.

Engineering Highlights

Rather than depending solely on model prompting, we introduced several architectural techniques to improve film quality:

  • Reference-Based Scene Continuity: Character portraits and environment references are generated first and reused throughout production. For consecutive scenes in the same location, Director Desk extracts the last frame of the previous scene and reuses it as the Image-to-Video reference, significantly improving visual continuity without additional generation cost.
  • Production Graph Orchestration: Assets, scenes, posters, trailers, and localized outputs are connected through a dependency graph, allowing downstream production stages to stay synchronized as projects evolve.
  • Smart Speaker Casting: During localization, dialogue is grouped into consistent speakers, translated with Qwen, and assigned language-specific voices before FFmpeg assembles the final dubbed soundtrack.
  • Accessibility Pipeline: Audio Description tracks are automatically mixed using FFmpeg sidechain ducking, lowering background audio only while narration is active to improve accessibility without manual audio engineering.
  • Efficient AI Orchestration: We reduced unnecessary LLM calls by replacing lightweight reasoning tasks with deterministic algorithms where appropriate, preserving API tokens for creative generation while improving overall system efficiency.

Challenges we ran into

1. Maintaining Character Continuity Across Scenes

Current open-source video generation models do not maintain persistent character identities across multiple shots. They have no built-in concept of recurring actors, visual memory, or scene continuity. Prompt engineering alone was insufficient. We redesigned our pipeline to generate clean character references first and, for consecutive scenes in the same location, reuse the final frame of the previous shot as the Image-to-Video reference for the next scene. This significantly improved visual continuity without consuming additional generation credits.

2. AI Models Interpret Images Very Differently Than Humans

Our initial approach created a production reference board by combining character portraits, environment images, and annotations into a single canvas. While it resembled a filmmaker's mood board, the video model interpreted the collage as one image to animate, producing inconsistent and distorted results. Replacing the collage with isolated character references became one of the biggest quality improvements in the project.

3. Balancing AI Capability with Lightweight Infrastructure

Running large local translation and speech models on a lightweight 2 vCPU / 2 GB production server quickly exhausted system memory during testing. Rather than increasing infrastructure requirements, we redesigned the localization pipeline to leverage Qwen for translation while keeping media processing lightweight. This allowed Director Desk to remain deployable on modest cloud infrastructure while reserving heavy computation for services better suited to those workloads.

4. Synchronizing Multilingual Dialogue with Video

Generating a dubbed soundtrack is more than translating subtitles. Every spoken line must begin at the correct timestamp while preserving natural pauses between speakers. We built a timing-aware assembly pipeline that generates speech clip-by-clip, automatically inserts silent gaps where necessary, and reconstructs a synchronized audio timeline before FFmpeg produces the final localized movie.

5. Orchestrating Long-Running AI Workflows

Film production is inherently sequential—later scenes often depend on assets produced earlier, while AI video generation itself is asynchronous and may take several minutes per shot. Designing a responsive user experience required separating orchestration from execution using background workers, real-time progress streaming, and dependency-aware production stages so creators always know what the system is doing without blocking the entire workflow.

Accomplishments that we're proud of

  • Building a true AI Showrunner Studio: Rather than creating another prompt-to-video tool, we built an end-to-end filmmaking platform that guides creators from a single idea through pre-production, production, post-production, localization, and release.

  • Solving continuity without retraining models: We designed a lightweight continuity pipeline using reference assets and last-frame scene transitions that significantly improves cross-scene visual consistency without additional model training or generation cost.

  • Creating a complete localization pipeline: Director Desk automatically generates subtitles, multilingual dubbing, and accessibility-focused Audio Description (AD) tracks, making AI-generated films easier to distribute to broader audiences.

  • Designing production tools beyond AI generation: We built an integrated timeline editor, Release Studio, poster designer, trailer generator, and asset management system so creators can finish and prepare films without leaving the platform.

  • Engineering for real-world deployment: Through modular orchestration, background processing, and lightweight cloud architecture, Director Desk remains practical to deploy while coordinating multiple AI models and media processing pipelines into a single production workflow.

What we learned

  • Great AI products are built around models, not on top of them. We started by experimenting with prompts, but quickly realized the real challenge was orchestration. Continuity, localization, editing, accessibility, and release all required carefully designed pipelines that connected multiple AI models into a single filmmaking workflow.

  • Work with model limitations instead of fighting them. Current video generation models struggle with temporal consistency. Rather than relying on increasingly complex prompts, we designed architectural solutions such as reference-based generation, last-frame scene continuation, and consistent speaker mapping that significantly improved output quality without additional generation cost.

  • Every API call matters. Working within a limited token budget forced us to evaluate every LLM request. Lightweight tasks were replaced with deterministic algorithms where possible, allowing us to reserve AI calls for creative reasoning while improving both efficiency and scalability.

  • Long-running AI workflows require thoughtful orchestration. Film production combines sequential dependencies with asynchronous generation. Building a responsive experience meant separating orchestration from execution using background workers, dependency-aware production stages, and real-time progress updates.

  • An AI video generator isn't an AI filmmaking platform. The biggest lesson was that generating scenes is only one part of filmmaking. Editing, localization, accessibility, continuity, and release preparation are equally important, and integrating them into a unified studio experience creates far more value than simply producing another AI-generated clip.

What's next for Director Desk

Director Desk has evolved into an end-to-end AI filmmaking studio, but we see this as the foundation rather than the finish line. Our roadmap focuses on making AI-generated films more natural, collaborative, and production-ready.

  • AI Lip Synchronization: Integrate post-processing models such as Wav2Lip or SadTalker to synchronize character lip movements with dubbed dialogue, creating more believable performances.

  • Advanced Multi-Character Scene Generation: Generate multiple actors independently while preserving their identities, then intelligently compose them into a single cinematic scene for richer storytelling.

  • Custom Voice Cloning: Allow creators to upload short voice samples to create consistent multilingual voice identities across an entire production.

  • Model-Agnostic AI Providers: Introduce a provider abstraction layer so creators can seamlessly switch between Alibaba Cloud DashScope, Runway, Pika, Kling, or future open-source models without changing their workflow.

  • Collaborative Cloud Studio: Transform Director Desk into a collaborative production environment where writers, editors, artists, and producers can work together on the same project with shared assets, version history, and production workflows.

Ultimately, our vision is to make Director Desk the operating system for AI filmmaking—bringing the complete journey from idea to final release into one intelligent, collaborative production studio.

Built With

Share this project:

Updates