RBS AIPRODUCER

11 specialized agents orchestrating your entire pipeline

Inspiration

As a developer, writer, and AI video creator, I observed a widespread barrier holding back creators trying to break into long-form productions and series: the blatant loss of visual consistency from shot to shot, combined with the time-consuming nightmare of manually stitching isolated tools together. Most existing workflows treat production steps as disconnected fragments, forcing creators to spend hours on repetitive manual integration. This project was born from the urgent need to solve this industry friction—shifting the focus from single-clip generation to full workflow automation, saving creators days of manual labor, and establishing a true, end-to-end production orchestrator built for sustainable storytelling.

What it does

RBS AIProducer is a comprehensive AI Showrunner designed to generate vertical and horizontal short movies and short drama series from a single raw idea. It orchestrates 11 specialized agents (Showrunner, Scriptwriter, Casting, Storyboard Artist, Shot Director, etc.) working as a continuous chain. It maintains full version history and supports a hybrid autonomy model: it can execute the entire pipeline automatically if all automated paths are selected, or pause at each stage for human evaluation, validation, and iteration. This step-by-step control optimizes API credit consumption and ensures strict quality control during this MVP phase, ultimately leading to a fully assembled final video cut.

How we built it

The application is entirely powered by the Alibaba Cloud ecosystem via the DashScope SDK. The architecture relies on a FastAPI backend (SQLAlchemy, SQLite) featuring a custom WorkflowEngine to steer the agent pipeline, alongside a centralized ToolRouter for model dispatching: Qwen3-Max for text reasoning, qwen3.7-plus for vision tasks (serving the Prompt Director role), Qwen-Image-2.0-Pro for storyboarding, and the Wan2.2/2.7 and HappyHorse suites for high-fidelity image and video generation.

Challenges we ran into

The biggest challenge was breaking the "black box" effect of generative tools and preventing error propagation down the pipeline. We overcame this by designing a strict human-in-the-loop validation framework between key production milestones and introducing a novel "Prompt Director" pattern to anchor vision models to actual visual assets rather than vague textual assumptions.

Accomplishments that we're proud of

Developing a 100% functional MVP pipeline, successfully deployed on an Alibaba Cloud server, that transforms a single story concept into a fully structured, visually cohesive episode with zero manual stitching.

What we learned

We mastered the nuances of the DashScope API and deep visual context management—learning to leverage multi-image vision inputs (feeding multiple reference images into Qwen vision models in a single call) to effectively mitigate identity drift and guarantee visual continuity.

What's next for RBS AIProducer

While the end-to-end pipeline is fully functional, we view this MVP as a strong foundation with clear paths for deep refinement. Our immediate roadmap focuses on elevating the quality of each agent and expanding creative control:

Specialized Prompt Engineering Agent: Introducing a dedicated agent to dynamically craft and optimize prompts specifically tailored to the conventions of Wan and HappyHorse, ensuring professional-grade visual consistency and reducing the need for manual prompt tuning.
Deep Character Profiling & Audio Integration: Expanding the Casting agent beyond visual traits to capture deeper psychological nuances. We also plan to integrate voice cloning and audio profiles for each character, allowing their specific voices to directly influence the emotional tone and lip-sync accuracy of downstream video shots.
Studio Module & Granular Control: Fully wiring the pre-scaffolded "Studio" module into the live pipeline. This will enable true hybrid workflows where creators can make granular, human-in-the-loop refinements to specific assets (e.g., tweaking a single shot's lighting) without triggering a full pipeline regeneration.
Format Flexibility: Expanding native support beyond vertical (9:16) short dramas to include horizontal (16:9) cinematic formats and broader aspect ratios to accommodate diverse platform requirements.

We are highly motivated to tackle these challenges and continue evolving RBS AIProducer into the ultimate AI-driven production studio.