What it does

AuteurOS is a Multi-Agent Virtual Production Studio that autonomously coordinates specialized AI agents to plan, shoot, edit, and assemble cinematic scenes.

How we built it

We designed specialized agents for different aspects of filmmaking, including screenplay, cinematography, dialogue, Foley, editing, and production coordination. Each agent operates within strict domain constraints and passes structured outputs to the next stage of production.

Challenges we ran into

Maintaining visual and narrative continuity across multiple shots required strict prompt conditioning and automated pairwise continuity checks.

Timeline Serialization: Translating dynamic multi-agent decisions into strict, industry-compliant OpenTimelineIO schemas without dropping timecodes.

Accomplishments that we're proud of

Zero-human-intervention assembly of a 12.5-second, 3-shot film scene with synchronized dialogue and layered Foley stems.

Full OpenTimelineIO (.otio) and Edit Decision List (.edl) export support for instant DaVinci Resolve import.

A transparent, auditable agent reasoning panel that visualizes the production crew's decisions in real time.

What we learned

Autonomous agents excel when given strict, specialized roles and domain constraints, such as cinematic camera grammar and timecode math, rather than being asked to "make a movie" in a single prompt.

What's next for AuteurOS

Multi-speaker dynamic lip-sync alignment.

Real-time 3D camera blocking via virtual viewport integration.

Collaborative multi-user writers' room overrides.

Built With

  • clickhouse
  • fastapi
  • ffmpeg
  • google-gemini
  • langgraph
  • mcp-clickhouse
  • nextjs
  • opentimelineio
  • python
  • react
  • tailwind-css
Share this project:

Updates