Clipper

Turn long, speech-heavy video into a first cut whose AI decisions and source timestamps stay inspectable.

Comment

Clipper: an auditable first cut for speech-heavy video

Turn long, speech-heavy video into a reviewable first cut whose AI decisions and source timestamps remain inspectable.

Live product: https://clip-my-gpt.pages.dev/
Source: https://github.com/seesianga/clip-my-gpt

Inspiration

AI video tools can produce clips quickly, but speed is not the only thing an editor needs. A first cut becomes difficult to trust, review, or hand off when the source evidence disappears and nobody can explain why each moment was selected.

We wanted to explore a different product question:

What if AI could help make the first cut without taking control away from the editor?

That led to Clipper, a video-review workflow for content producers and editors working with interviews, podcasts, presentations, and other speech-heavy footage. Instead of treating the model as an autonomous editor, Clipper gives it a narrower role: interpret the transcript against an editorial brief, while deterministic application code preserves timestamps, selects exact source intervals, and renders the result.

The goal is not simply to generate a short video. It is to produce a first cut that another person can inspect, challenge, revise, and confidently hand off.

What Clipper does

A user provides an authorized local video or eligible public video URL, selects GPT-5.6 Luna or Terra, chooses a target duration between 30 and 120 seconds, and completes a structured editorial brief.

Clipper then:

  1. Stores the source privately.
  2. Extracts and transcribes bounded audio chunks.
  3. Sends immutable transcript segments and the editorial brief to GPT-5.6.
  4. Validates the model’s structured annotations.
  5. Deterministically ranks and selects non-overlapping source intervals.
  6. Renders those exact intervals into an H.264/AAC MP4 with FFmpeg.
  7. Preserves the transcript evidence, scores, explanations, warnings, model provenance, and selected intervals alongside the result.
  8. Lets the editor reorder or omit selected intervals without changing their source timestamps.
  9. Exports the current sequence as OpenTimelineIO together with an original-versus-current decision diff.

For the premium path, Clipper uses the disclosed ranking formula:

$$

\text{rank score}

0.65 \times \text{editorial suitability} + 0.35 \times \text{transcript intensity} $$

If the available non-overlapping material cannot fill the requested duration, Clipper returns a shorter cut with an explicit underfill warning. It does not silently duplicate footage to reach the target.

Why GPT-5.6 matters

GPT-5.6 is decision-relevant rather than decorative.

Luna or Terra interprets each transcript segment against the complete editorial brief and returns bounded fields such as editorial suitability, narrative role, intensity, confidence, and an explanation. Valid changes to those fields can change which source intervals the deterministic selection code chooses and how it orders them.

We added a controlled sensitivity test that holds timestamps, transcript text, sentiment, confidence, and source duration constant. Under the sentiment-only condition, the application selects one pair of segments. Restoring the GPT-generated editorial-suitability and narrative-role fields changes the selected edit. This demonstrates that the validated model output can causally influence the cut rather than merely generating commentary beside it.

The model’s authority is deliberately limited. GPT-5.6 does not:

  • transcribe the source;
  • create or alter timestamps;
  • see video frames;
  • choose arbitrary frames;
  • bypass schema validation;
  • render the video; or
  • publish anything autonomously.

Application code owns the source intervals and edit constraints. FFmpeg owns rendering. Human editors retain the final decision.

How we built it

Clipper combines:

  • React 19 and TypeScript for the product interface;
  • Cloudflare Pages for the static frontend;
  • a same-origin Pages Function connected to a private Worker;
  • Cloudflare Workflows for resumable analysis and human rerenders;
  • private R2 storage for sources, checkpoints, results, and rendered media;
  • D1 for authentication, entitlements, audit records, and usage accounting;
  • Workers AI Whisper for transcription;
  • the OpenAI Responses API with GPT-5.6 Luna or Terra;
  • strict Structured Outputs with runtime validation and store: false;
  • isolated Cloudflare Containers running FFmpeg;
  • Daytona sandboxes for durable credential-free public-video imports; and
  • OpenTimelineIO plus a decision-diff format for editor handoff.

Long-running work is divided into durable stages. Checkpoints allow a retry to reuse completed work instead of automatically repeating an expensive provider call. Before each GPT-5.6 attempt, D1 atomically reserves a conservative token allowance against the account’s daily ceiling. Successful responses reconcile that reservation with reported usage, while ambiguous states fail closed.

Browser media and analysis routes require the authenticated owner. Sources and results remain private in R2 behind protected routes, and cross-account or ownerless browser access fails closed.

The current deployed application is bound to annotated release tag build-week-final-application-2026-07-21-v17, commit ebb0da0cbd073f879cdab36e13768ff7ab0e0288. A no-store release manifest records the repository, branch, source tree, tool versions, lockfile, and hashes for all served files.

At that exact application source, the full local verification gate completed 385 tests:

  • 384 passed;
  • 0 failed;
  • 1 expected local SQLite skip;
  • lint and type-checking passed;
  • both application builds passed;
  • all three retained evidence evaluators passed;
  • npm audit reported zero vulnerabilities; and
  • the backend Worker deployment dry run passed.

How Codex accelerated development

We used Codex with GPT-5.6 Sol as a repository-aware engineering partner across the stack.

Codex helped us:

  • map the existing architecture and trace cross-layer behavior;
  • implement and review Worker, Workflow, R2, D1, and Container paths;
  • build validation and failure-handling boundaries;
  • create regression tests for security and reliability cases;
  • reason about checkpoint reuse and duplicate-provider-call risks;
  • verify media ownership and capability rules;
  • improve import-error classification and recovery guidance;
  • build source-to-deployment release verification; and
  • identify contradictions between code, documentation, deployment evidence, and submission materials.

Codex accelerated implementation and investigation, but it did not autonomously decide the product. Human decisions defined the target user, problem, model authority, ranking formula, privacy policy, access model, acceptable claims, deployment approval, and final release acceptance.

Development-time GPT-5.6 Sol use through Codex is separate from Clipper’s runtime GPT-5.6 Luna and Terra calls.

Challenges we faced

Keeping the model useful without giving it unsafe authority

The easiest implementation would have been to ask a model to “edit the video” and trust the result. That would also make the workflow difficult to validate.

We instead separated semantic judgment from media authority. GPT-5.6 can annotate meaning, but immutable segment identifiers, timestamps, interval constraints, and rendering remain application-controlled.

Making long-running media work recoverable

Transcription, model annotation, rendering, storage, and cleanup can outlive a normal request. We used durable Workflows and private checkpoints so recoverable work can resume without starting from zero.

The difficult boundary was handling a successful provider response whose checkpoint write becomes uncertain. We chose conservative accounting and fail-closed behavior rather than risking an uncontrolled duplicate call.

Protecting private media

Uploads, imported sources, analysis results, and revised renders needed consistent ownership rules. We implemented authenticated ownership checks, scoped capabilities, protected delivery, rate limits, cleanup verification, and private storage.

Handling public-video imports honestly

Public video sources can fail because of HTTP 403 or 429 responses, bot challenges, unavailable formats, DRM, login requirements, or provider-specific egress behavior. Clipper does not forward browser cookies or bypass those controls.

We built bounded diagnostics and recovery guidance so an upstream challenge is not incorrectly described as a private video or a user authentication failure. Local upload remains the reliable fallback.

Keeping release evidence synchronized

A successful build or HTTP 200 response does not prove that the live application matches the reviewed source. We created a release manifest and health contract that bind the deployed frontend and Worker to an exact commit.

This also taught us that application identity, runtime evidence, browser evidence, media QA, repository access, and submission state are different proof boundaries. One cannot silently substitute for another.

What we learned

The most important lesson was that trustworthy AI products are often built by reducing model authority, not expanding it.

We also learned that:

  • deterministic post-processing makes model behavior easier to test and explain;
  • provenance is a user-facing product feature, not merely debugging metadata;
  • an honest underfill warning is better than manufacturing footage to satisfy a target;
  • human revision should create a new version rather than silently overwrite the original;
  • infrastructure reliability becomes part of the product when users wait for long-running media work;
  • runtime GPT use and development-time Codex use must be explained separately; and
  • evidence should state exactly what it proves—and what it does not.

What we are proud of

Clipper goes beyond a thin model wrapper. It combines bounded semantic reasoning, deterministic interval selection, protected media processing, durable recovery, human revision, and an editor-oriented handoff.

We are especially proud that the result remains inspectable from beginning to end:

immutable transcript → validated GPT-5.6 annotations → deterministic ranking → exact source intervals → rendered MP4 → versioned human revision → OpenTimelineIO + decision diff

The no-account sample is explicitly labeled as a precomputed synthetic fixture. Opening it does not upload media, start an analysis job, or call GPT-5.6. It demonstrates the decision contract and interface without pretending to be live-model evidence.

What is next

The next product evaluation will compare Clipper with existing manual and AI-assisted workflows using:

  • time to first usable cut;
  • number of corrections;
  • source-grounding accuracy;
  • task-completion rate; and
  • reviewer confidence.

We also want to validate OpenTimelineIO import, relinking, and round-trip behavior in named editing applications, expand browser and assistive-technology testing, and collect current-release runtime evidence across multiple authorized sources.

Clipper does not yet claim measured time savings, model superiority, adoption, production scale, or user-impact results. Those claims should come only after controlled evaluation.

The current public sample also requires explicit fixture-distribution authorization before it is described as rights-cleared or submission-ready.

Built With

  • cloudflare-containers
  • cloudflare-d1
  • cloudflare-durable-objects
  • cloudflare-pages
  • cloudflare-pages-functions
  • cloudflare-r2
  • cloudflare-rate-limiting
  • cloudflare-workers
  • cloudflare-workers-ai
  • cloudflare-workflows
  • ffmpeg
  • gpt-5.6-luna
  • gpt-5.6-terra
  • next.js
  • openai-responses-api
  • opentimelineio
  • react-19
  • typescript
  • vinext
  • vite
Share this project:

Updates