-
-
Structured narrative generation
-
Thumbnail Generation
-
Timed voice generation and media processing
-
Brand and collaboration discovery
-
Audio/video synchronization
Inspiration
Generative AI has made it incredibly easy to create individual pieces of content. You can generate a script in one tool, search for visual references somewhere else, create images with another model, generate a voiceover on a separate platform, and translate it using yet another service.
But producing a video is not a collection of isolated AI tasks. It is a workflow.
Creators still have to decide what happens next, keep track of dependencies, research references, understand visual language, determine whether an idea is actually shootable with their available equipment, manage audio and localization, and deal with copyright and compliance concerns.
We wanted to solve the coordination problem.
That led us to CreatorCrew, an AI-native production studio where the creator simply expresses intent in natural language and specialized agents coordinate the production process behind the scenes.
Instead of asking creators to operate a collection of AI tools, we wanted to build a system where:
Creators express intent. Agents coordinate the workflow. Deterministic tools handle production.
What it does
CreatorCrew turns a natural-language creative idea into a coordinated production workflow.
A creator can start with:
"Create a futuristic cyberpunk coffee brand advertisement."
CreatorCrew can then orchestrate:
- Script generation with structured beats, timing, visual intent, and audio intent.
- Live visual research using Parallel to discover relevant cinematic references from the web.
- Multimodal reference analysis using Google Gemini Vision to understand framing, camera language, lighting, composition, color, and mood.
- Visual grammar synthesis to convert multiple references into consistent stylistic rules.
- Storyboard generation with structured shot plans.
- Production planning that considers the creator's actual equipment, location, crew, and constraints.
- AI keyframe generation
- Timed AI voiceover using Gemini TTS and FFmpeg.
- Cultural dubbing that combines Parallel research, Gemini reasoning, and TTS to localize content.
- Creator scouting for brand deals and collaboration opportunities using live web intelligence.
- Media synchronization for creator-uploaded video and audio.
- Compliance and clearance with Green/Yellow/Red decisions and human approval gates.
The entire system is connected through a stateful workflow DAG, meaning agents do not operate independently in isolation. Each stage understands what has already been completed, what is currently available, and which downstream stages are ready to execute.
Creator-Controlled Production
CreatorCrew is designed to assist creators, not replace their creative decisions.
The workflow is intentionally flexible. A creator can:
- Follow the suggested production flow from script to final assets.
- Ask the system to regenerate any individual stage if the result does not match their vision.
- Change the creative direction or underlying idea at any point and continue from the updated state.
- Give additional natural-language instructions to refine scripts, storyboards, audio, localization, or production plans.
- Allow the agents to suggest the next step while retaining complete creative control over what happens next.
This means CreatorCrew does not impose a fixed generation pipeline. The Workflow DAG manages dependencies and execution safety, while the creator remains in control of the creative direction.
Bring Your Own Voice
CreatorCrew is also not restricted to AI-generated voiceovers.
Creators can choose between:
- AI Voice: Generate a timed voiceover using Gemini TTS.
- Creator Voice: Upload their own recorded video or audio and use it as the source for the production workflow.
For creator-uploaded media, the Media Sync Agent uses FFmpeg/FFprobe to analyze and align the media with the project's timing and downstream workflow requirements.
This gives creators the flexibility to use AI when they want speed and experimentation, while retaining their own voice, identity, and authenticity when that matters.
AI generates possibilities. The creator decides what stays.
How we built it
CreatorCrew is built around a multi-agent orchestration architecture.
The frontend is built with Next.js 14, React, and Tailwind CSS, while the backend uses FastAPI, Python 3.12, Pydantic, and an asynchronous background worker system.
At the center is our workflow engine:
Creator
│
▼
Natural Language Chat
│
▼
Intent Router
│
▼
Workflow DAG + State Machine
│
├── Script Agent
├── Storyboard Agent
├── Audio Agent
├── Cultural Dubbing Agent
├── Creator Scout Agent
├── Media Sync Agent
└── Compliance Agent
Google Gemini as the reasoning layer
Gemini is deeply integrated throughout CreatorCrew rather than being used as a single chatbot.
We use Gemini for:
- Structured script generation.
- Pydantic-compatible structured outputs.
- Multimodal analysis of extracted video frames.
- Camera, composition, lighting, color, and mood analysis.
- Visual grammar synthesis.
- Production constraint reasoning.
- Storyboard adaptation.
- Cultural localization and transcreation.
- Compliance reasoning.
- Voice generation through Gemini TTS.
The storyboard pipeline is particularly multimodal:
Script
↓
Parallel Reference Search
↓
Parallel Content Extraction
↓
FFmpeg Frame Extraction
↓
Gemini Vision Analysis
↓
Visual Grammar
↓
Structured Shot Planning
↓
Production Constraint Planning
↓
Adaptation
↓
Image Generation
Parallel for live web intelligence
We use Parallel Web Systems as the real-time research layer for our agents.
For visual production, Parallel searches the live web for relevant cinematic references and helps extract useful web/video content before Gemini analyzes the resulting frames.
For cultural dubbing, Parallel helps research regional language and cultural context before the content is transcreated.
For creator scouting and clearance workflows, Parallel provides access to current external information that a static LLM knowledge base cannot reliably provide.
This creates a powerful division of responsibilities:
Parallel finds what is happening in the world. Gemini reasons about what it means.
Deterministic media infrastructure
AI reasoning is combined with deterministic production tools wherever precision matters.
We use:
- FFmpeg / FFprobe for media processing, duration analysis, audio alignment, and synchronization.
- PostgreSQL for deployed persistence.
- SQLite for local development.
- Pydantic for structured contracts between agents.
- FastAPI for the service layer.
- Next.js for the creator-facing studio.
The project also includes a human-in-the-loop compliance system. Potentially sensitive operations can pause the workflow at a Yellow checkpoint, allowing a creator to review and approve the action before the DAG continues.
Challenges we ran into
Building an actual multi-agent production workflow introduced challenges beyond simply calling an LLM API.
1. Coordinating independent agents
Each agent was developed around its own responsibilities and data models. The difficult part was making them behave as one system.
We had to design shared project state, structured schemas, stage dependencies, background jobs, and a workflow DAG so that one agent could reliably consume another agent's output.
2. Keeping AI outputs deterministic enough for downstream systems
Natural-language generation is flexible, but production pipelines need predictable data.
We therefore used structured outputs and Pydantic models throughout the workflow so that scripts, shots, production plans, audio metadata, and other artifacts could be passed between agents safely.
3. Bridging web intelligence and multimodal AI
Finding a useful reference is only the first step.
We had to build a pipeline that could:
- Search for relevant references.
- Extract usable media.
- Process the media with FFmpeg.
- Extract representative frames.
- Send those frames to Gemini Vision.
- Aggregate the observations into visual grammar.
- Feed that grammar into structured storyboard generation.
This was one of the most technically involved parts of the project.
4. Production realism
A visually impressive storyboard is not necessarily shootable.
We wanted CreatorCrew to reason about constraints such as:
- Smartphone vs. cinema camera.
- Available lenses.
- Tripod vs. gimbal.
- Solo creator vs. multi-person crew.
- Indoor vs. outdoor locations.
- Available lighting equipment.
The production planner therefore evaluates each shot against the creator's actual constraints and can flag shots as high, medium, or low feasibility rather than blindly generating an idealized production plan.
5. Managing long-running workflows
Research, multimodal analysis, image generation, TTS, and media processing can all take significant time.
Instead of making the user wait on one giant request, we built a persistent background job workflow with project state and job status tracking. This allows the UI and API to understand which stages are queued, running, completed, or blocked.
6. External service limitations
During development we also encountered API quotas and provider limitations, particularly around generative image inference. This reinforced an important architectural decision: external model failures should be isolated to individual stages rather than bringing down the entire production workflow.
Accomplishments that we're proud of
We are particularly proud that CreatorCrew evolved from a collection of independent agents into a functioning production system.
🤖 True multi-agent orchestration
The agents are not simply multiple prompts behind a UI. They operate through a shared project state and workflow DAG with explicit dependencies and background execution.
🔎 Deep Parallel integration
Parallel is used as a live intelligence layer across visual research, cultural research, creator scouting, and clearance-oriented workflows.
🧠 Deep Gemini integration
Gemini powers multiple reasoning and multimodal stages, including structured generation, visual analysis, production reasoning, compliance, localization, and TTS.
🎥 End-to-end multimodal pipeline
We connected:
web research → video extraction → frame analysis → visual grammar → storyboard → production planning → image generation
into a single pipeline.
🛡️ Human-in-the-loop governance
CreatorCrew does not blindly automate sensitive workflows. Our Green/Yellow/Red compliance model allows the system to pause and request human approval when required.
🌍 Production localization
The dubbing workflow goes beyond literal translation by researching cultural context before generating localized audio.
🚀 Full-stack deployment
The system is deployed as a complete application with a Next.js frontend, FastAPI backend, persistent database, background processing, API documentation, and production-ready workflow routes.
🎨 Creator-in-the-loop flexibility
CreatorCrew is built around creative control rather than forced automation. Creators can follow the recommended workflow, regenerate individual stages, change their creative direction, or provide new instructions at any point.
The system preserves workflow state and dependencies while allowing the creator to remain the final decision-maker.
🎙️ Creator Voice + AI Voice
The audio workflow supports both AI-generated voiceovers and creator-provided media. A creator can upload their own voice/video instead of being forced into an AI-generated voice pipeline, enabling a balance between automation and authentic creator identity.
What we learned
The biggest lesson was that building an agent is much easier than building a system of agents.
The difficult engineering problems were not only model prompts. They were:
- Designing reliable contracts between agents.
- Managing shared state.
- Handling asynchronous jobs.
- Enforcing workflow dependencies.
- Recovering from external API failures.
- Combining deterministic tooling with probabilistic AI.
- Knowing when automation should stop and ask a human.
We also learned that multimodal AI becomes significantly more useful when it is grounded in external information.
Instead of asking Gemini to invent what a cinematic reference might look like, CreatorCrew can use Parallel to discover real references, extract their visual information, and then use Gemini to reason over the actual frames.
This separation of responsibilities made the system much more powerful:
Search with Parallel. Reason with Gemini. Execute with deterministic tools.
Most importantly, we learned that the future of creative AI is not necessarily another standalone generation tool. There is enormous value in the orchestration layer that connects many capabilities into one coherent workflow.
What's next for CreatorCrew
CreatorCrew currently focuses on the core production workflow, but the architecture gives us a foundation for a much broader creator operating system.
Our next steps include:
- Full video assembly, turning generated shots, audio, and assets into complete edited videos.
- Smarter adaptive workflows, where the DAG automatically recalibrates when creators change scripts or miss production milestones.
- More advanced creator profiles, allowing agents to learn a creator's preferred visual style, equipment, audience, and brand identity.
- Expanded brand collaboration workflows, connecting creator scouting with campaign requirements and deliverables.
- Deeper rights and clearance intelligence for music, imagery, brands, and likeness.
- More multilingual voice and localization capabilities.
- Improved model fallback and cost-aware routing across different AI providers.
- Collaboration features for teams, editors, producers, and clients.
Our long-term vision is for CreatorCrew to become more than a content generator.
We want it to become an AI production crew that understands the creative intent, researches the world, plans the production, executes the repetitive work, and knows when to hand control back to the human creator.
Log in or sign up for Devpost to join the conversation.