CreatorPilot

AI agents turn trends into publish-ready videos end-to-end—discovering ideas, planning storyboards, assembling media, rendering content, and optimizing metadata with minimal human input.

Comment

Creator Pilot

What Inspired This Project

Modern content creation is still a long, manual pipeline: research trends, ideate, gather assets, edit, render, write metadata, publish, repeat.

CreatorPilot explores whether that pipeline can become multi-agent and autonomous—where agents coordinate across tools and outputs, not just one prompt at a time.

What I Learned

  1. Agents work best when they orchestrate tools rather than generate everything themselves. Instead of asking a model to do everything in one step, the system is more reliable when broken into stages and orchestrated.

  2. Creative pipelines require deterministic components. AI generates ideas and text, but rendering, file management, and media handling need deterministic systems.

  3. The biggest challenge is orchestration, not generation. Generation is easy; coordinating assets, story structure, rendering, and publishing outputs is the hard part.

How I Built It

CreatorPilot is a modular multi-agent pipeline implemented inside a Next.js monolith (no external orchestration infrastructure).

A central Orchestrator Agent owns shared workflow state and decides which specialist agent runs next. A Profile / Memory Agent (Prisma-backed local storage) loads creator preferences, source settings, recent renders, and publishing history.

Core agents

  • Orchestrator Agent: coordinates trend discovery, ideation, storyboarding, rendering, metadata generation, and publishing workflows.
  • Profile / Memory Agent: loads creator profile data and history from Prisma-backed local storage.
  • Trend Discovery Agent: syncs enabled sources, fetches RSS entries, clusters related trends, and ranks them for creator fit.
  • Content Ideation Agent: converts trends or uploaded creator media into structured video concepts, hooks, and narrative outlines.
  • Media Selection Agent: resolves uploaded creator assets and prepares them for downstream planning and rendering.
  • Storyboard Agent: builds beat-by-beat coverage plans, evaluates whether existing media is sufficient, and prepares generated preview support when needed.
  • Render Agent: assembles narration, overlays, subtitles, and support media into FFmpeg-based video variants.
  • Metadata Agent: generates YouTube-ready titles, descriptions, tags, captions, and posting recommendations.
  • Publishing Agent: validates the selected render, checks readiness for upload, and performs live or mock YouTube publishing.

Challenges I Ran Into

1. Managing multi-agent workflow state

State needs to persist across steps so specialist agents can use consistent inputs and outputs.

2. Managing media pipelines

Different media formats and asset sources need consistent handling so downstream rendering doesn't break.

3. Coordinating agent decisions

Actions like storyboarding, metadata, and publishing depend on decisions made earlier in trend discovery and ideation.

Why CreatorPilot Matters

CreatorPilot demonstrates how AI agents can move beyond chat interfaces and become active collaborators in creative workflows.

Instead of generating isolated outputs, the system acts as a production assistant that plans, generates, and assembles a full video package end-to-end.

What's next for CreatorPilot

CreatorPilot is now a local-first single-instance prototype with explicit orchestration, specialized agents, creator memory, persisted render outputs, and visible agent-level workflow logging. The current implementation is intentionally monolithic and demo-oriented, which makes it practical for rapid iteration and hackathon presentation. Logical next steps include:

  • Multi-user and session-aware isolation instead of the current single local-user workflow
  • Stronger memory summarization and performance-aware feedback from completed renders and publishing outcomes
  • Smarter orchestrator decision policies for choosing between trend-led and media-led strategies
  • Better ranking signals for idea quality, render quality, and publishing recommendations
  • Richer observability for agent decisions, tool usage, and failure recovery
  • Optional distributed execution if render or upload workloads eventually outgrow the current in-process job model

Built With

Share this project:

Updates