Dailies

Dailies transforms raw footage into a reviewed video cut through an autonomous, self-correcting post-production workflow.

Comment 1

About the Project

Inspiration

Video editing requires many interconnected decisions. A creator must diagnose weak footage, plan a coherent cut, select visual and audio treatments, render the result, review it, and revise it. Most artificial intelligence tools assist with only one of these steps. They generate text or isolated assets, but they leave the creator responsible for coordinating the entire workflow.

Dailies was created to complete that workflow. It gives a creator a practical starting point from raw footage and continues working until it produces a reviewed cut. The system can also use the creator's own YouTube retention history to inform editing decisions when the creator chooses to connect that data.

The name comes from film production. "Dailies" are the newly recorded scenes that filmmakers review during production to evaluate the work and decide what to change next. The name fits this product because its agent repeatedly examines footage, renders a draft, reviews the result, and applies targeted revisions. Dailies turns that established review practice into an autonomous post-production loop for online creators.

What it does

A creator uploads their raw footage and selects one of two operating modes.

In Autonomous mode, Dailies completes the workflow without further input. It analyzes the footage, designs an edit, generates supporting assets, renders the cut, reviews the actual rendered video, and revises the draft when necessary. The review loop can produce up to three autonomous drafts, and the project cannot be marked complete until the reviewer passes a draft.

In Collaborative mode, the same workflow pauses after the edit plan and after a draft passes review. The creator can approve the work, provide feedback, or request a targeted change. The creator can also request revisions through chat after the project is complete. A revision resumes only the necessary planning, asset, rendering, and review stages instead of requiring a new submission of the entire project.

The workflow performs the following actions:

  1. It stores uploaded footage in a private Google Cloud Storage bucket.
  2. It uses Gemini multimodal analysis to identify shot structure, pacing issues, editorial opportunities, and audio direction.
  3. It uses a Google ADK LlmAgent to design the edit timeline and select appropriate visual and audio treatments.
  4. It generates music with Lyria 3 and can use Vertex AI Veo for an optional motion treatment.
  5. It renders the planned cut with FFmpeg.
  6. It reviews the rendered output with Gemini and applies a targeted revision if the draft does not pass.
  7. It checkpoints each stage in Firestore so interrupted work can resume from durable state.
  8. It can retrieve read-only YouTube Analytics data through OAuth and query normalized retention history in ClickHouse when the creator enables that optional integration.

Why it is agentic

Dailies operates beyond a standard chat loop. The creator provides a goal and source footage, but the system determines and executes the intermediate steps. Its editorial agent invokes scoped tools for analysis, planning, asset generation, rendering, review, and revision. The agent evaluates its own rendered output rather than assuming that a successful render is a successful edit.

The system also supports asynchronous work. Each project has a durable job record, progress checkpoints, an event ledger, lease-based recovery, and explicit review states. If a worker stops or is redeployed, another worker can resume the project from the last completed stage instead of restarting the full workflow.

Value proposition

Dailies removes the coordination burden from early post-production. It gives creators a complete first cut rather than a collection of disconnected suggestions or assets. Its autonomous mode reduces manual handoffs, while its collaborative mode preserves creator control at the two decisions that matter most: approval of the edit plan and approval of the reviewed draft.

The product also connects creation with learning. When a creator authorizes YouTube access, Dailies can incorporate normalized retention history into its editorial recommendations. This allows previous audience behavior to inform the next cut without making the core workflow dependent on a social platform connection.

How We Built It

Dailies uses a modular TypeScript architecture with three deployed service boundaries.

The public frontend and API run together in one Cloud Run service. This service hosts the React studio, authenticates the creator, owns project data, accepts uploads, and starts or resumes workflow jobs.

The private agent service runs the Google ADK editorial agent. The LlmAgent coordinates function tools that perform Gemini video analysis, edit design, optional retention queries, Lyria music generation, optional Veo generation, FFmpeg rendering, rendered-draft review, and revision. Tool responsibilities are isolated so the agent can make editorial decisions without receiving unrestricted infrastructure access.

The private ingestion service handles YouTube OAuth synchronization and controlled ClickHouse writes. The agent receives read-only access to retention evidence through the ClickHouse MCP interface. This separation prevents the editorial workflow from modifying analytics history.

Firestore stores projects, job state, checkpoints, review states, and the activity ledger. Cloud Storage stores private source media and generated assets. Service-to-service tokens protect the private services. Production authentication relies on verified Google Cloud identity headers, with Google OAuth available for deployments that do not use Identity-Aware Proxy.

Technologies used

  • Google Agent Development Kit for TypeScript
  • Google Gen AI SDK
  • Gemini multimodal models for analysis, planning, review, and selected asset tasks
  • Lyria 3 for original music generation
  • Vertex AI Veo for optional generated motion treatments
  • Vertex Text-to-Speech for chat read-aloud
  • Google Cloud Run for the application services
  • Google Cloud Firestore for durable state, checkpoints, and events
  • Google Cloud Storage for source footage and generated media
  • React, Redux Toolkit, Vite, Node.js, TypeScript, Express, Zod, and FFmpeg
  • ClickHouse Cloud and the ClickHouse MCP server for optional retention analytics
  • YouTube Analytics API with read-only OAuth scopes

Other data sources

The core editing workflow requires only footage supplied by the creator. If the creator explicitly connects a YouTube account, the ingestion service retrieves the creator's own retention data through the official YouTube Analytics API. The service normalizes that data before writing it to ClickHouse. Dailies does not scrape YouTube Studio, and the optional connection uses read-only scopes.

Challenges

Designing an agent that takes action

The central challenge was to avoid building a chatbot that merely describes an edit. Dailies needed to turn editorial decisions into generated assets and a rendered video. It also needed to inspect the result and revise it when the actual output did not satisfy the review criteria.

Recovering long-running work safely

Video analysis, generation, and rendering can outlast a single request or worker process. The workflow therefore records durable checkpoints and events in Firestore. Recovery logic distinguishes new work, autonomous retries, collaborative approvals, and targeted chat revisions.

Balancing autonomy with creator control

A fully autonomous workflow is efficient, but creators may want to approve the direction before expensive generation and rendering begin. The two operating modes share one pipeline while providing different control points. This preserves unattended execution without forcing every creator into the same interaction model.

Using external data responsibly

Retention history can improve recommendations, but access must be optional and properly authorized. Dailies uses official OAuth access, a separate ingestion service, and a read-only query boundary for the agent. The core workflow remains available without YouTube or ClickHouse.

Accomplishments

  • Dailies completes a real, multi-step post-production workflow instead of returning only advice.
  • The agent reviews the rendered artifact and can revise it before completion.
  • Autonomous and collaborative modes use the same durable pipeline without duplicating the implementation.
  • Firestore checkpoints allow interrupted jobs to resume from completed work.
  • Private services and scoped tools limit access to media, analytics, and infrastructure.
  • Optional Lyria 3 and Veo integrations add original music and motion treatments within the same editorial plan.
  • Post-completion chat requests can trigger targeted revisions without restarting the project.

Findings and Learnings

A useful creative agent must evaluate outputs, not only produce them. Rendering a file proves that the pipeline ran, but it does not prove that the edit is coherent. Adding a Gemini review stage changed the workflow from simple automation into a self-correcting system.

Durable state is also a product feature. Creators should not need to understand worker restarts, deployment events, or model latency. Checkpoints, leases, and event history make long-running creative work understandable and recoverable.

Finally, autonomy and collaboration do not need separate products. They can be two policies applied to the same state machine. Dailies can work unattended when speed matters and pause for approval when creative control matters.

What Is Next

  • Expand retention analysis across larger portions of a creator's catalog.
  • Add more explainable evidence to each editorial recommendation.
  • Improve cost and latency controls for generated motion treatments.
  • Add additional export formats and direct publishing workflows.
  • Extend collaborative review with comments tied to exact timestamps.

Built With

Share this project:

Updates