Inspiration
People don't watch YouTube the way they used to. A 20-minute video has to compete with a feed of 30-second clips that ask nothing of you, and more and more of the audience simply lives in that feed now — Reels, TikTok, Shorts. That's where people discover things, and increasingly it's where they get their information.
For creators, this is a strange trap. The long video is where the real work and the real value are, but the only way to get it in front of people is to also produce short-form versions of it: cut a hook, reframe it vertical, caption it, and post it as a trailer that points back to the full upload. Done by hand that's an hour or more per video of scrubbing through footage looking for the right 40 seconds. Most creators either skip it or pay an editor, and the long video quietly gets fewer views than it deserves.
I wanted to build the thing that closes that gap: give it the long video, get back a short-form trailer that makes people want to watch the whole thing — without giving away the ending.
What it does
Cliffhang takes a long video (a link or a file) and produces short-form content from it in two forms.
Trailer mode is the heart of it. It reads the whole video, works out where the payoff is — the answer, the reveal, the conclusion — and then builds a teaser from the material before that point: the setup, the questions, the moments where the stakes go up. It picks up to five segments whose tension rises from one to the next, ends on an open question, and never touches the climax. The result is a 20–60 second vertical clip with burned-in captions that works as a trailer for the full video.
Clips mode finds the strongest standalone moments and cuts each one into its own short, with a title and a one-line explanation of why it was chosen.
Both come out in a phone format (9:16, with the full frame preserved over a blurred background, the standard Shorts look) and a laptop format (16:9), from a simple web page: paste a link, press Run, watch the stages go by, review the results with the reasoning next to each cut.
How we built it
The pipeline is Python plus ffmpeg, and it runs entirely on your own machine.
Transcribe. Whisper (via faster-whisper) produces a word-level transcript, which I group into sentences on punctuation. Measure. For every sentence I compute signals from the audio and the text: loudness and loudness variation (z-scored against the whole video, so emphasis stands out), speech rate, the pause before the sentence, TF-IDF novelty against the previous few sentences (is something new being said), TF-IDF centrality against the whole video (is this on the main topic), and pattern flags for questions, numbers, hook phrasing, dilemma words, conclusion markers, and intro/outro boilerplate. Three weighted scores come out of that: attention, tension, and payoff. Find the climax. The video is split into beats at pauses. The climax is the beat in the second half where the payoff score peaks; it and everything after it are off-limits to the trailer. Edit. A dynamic program chooses up to five pre-climax beats, in chronological order, whose tension strictly increases, maximizing total tension under a length budget scaled to the source video — a weighted longest-increasing-subsequence with a knapsack constraint. Ending on a question or an unresolved "but…" earns a bonus. Adjacent beats merge into one continuous cut, so the tool decides per video whether the trailer is a single stretch or a montage. Check. The cut is verified: tension rises with no dips, the ending is open, nothing overlaps the climax, the length is near target. A failing cut is re-solved once with looser constraints. Render. ffmpeg trims each segment on exact timestamps, reframes it, burns in captions, adds tiny audio fades, and joins the segments as one continuous stream. A small web UI on top (standard library only) shows the pipeline stages, the log, and the results with the measured reason for every choice.
Challenges we ran into
Most of the pain was in the video plumbing, not the algorithm. Homebrew's ffmpeg ships without the subtitle renderer, so I built a caption path that draws every caption for a segment into a single sprite image with Pillow and uses a time-dependent crop expression to show the right one per frame. Then a test video downloaded with a video track that ended at six minutes while the audio ran the full 27, which sent me chasing a "no video stream" error through three layers before I probed the source file; the downloader now checks for that and re-fetches. And ffmpeg's seek options interact in ways that vary by version — one combination trimmed the audio five seconds short on every segment and padded it with silence — so trimming now happens inside the filter graph on absolute timestamps, which is exact.
On the algorithm side, the hardest part was getting clips to start in the right place. A sentence boundary isn't enough: "Alternatively, what if you do them…" is a complete sentence and a terrible opening line. Penalizing continuation openers and rewarding lines that follow a pause fixed most of it.
Accomplishments that we're proud of
The trailer editor actually works, and it works without a language model. I framed "cut a teaser that builds and stops before the payoff" as an optimization problem — choose segments in order with strictly rising tension, maximize total tension under a length budget, never enter the region after the climax — and solved it with a dynamic program over measured signals like loudness, pace, novelty, and phrasing. On a 27-minute video it found the payoff, picked five moments whose tension climbed 6.0 → 6.7 → 7.0 → 7.4 → 8.1, and cut a 54-second trailer that ends on an open question. Watching that come out of a scoring function and a DP rather than a prompt was the moment the project felt real.
Every decision the tool makes is explainable. Each clip and each trailer segment comes with the reason it was chosen in plain language — "louder than usual, introduces new material, sets up a dilemma, starts after a pause" — because those reasons are the actual numbers that drove the choice, not a story written afterward.
It runs end to end on a laptop and nothing leaves the machine: a real link goes in and real MP4s come out, in both phone and laptop formats, from a web page that someone who has never seen a terminal can use. And it survived contact with the messy parts — an ffmpeg build without a subtitle renderer, a download with a truncated video track, seeking behavior that differed by version — with fixes that make it more robust for the next video, not just the one that broke.
What we learned
That "what makes a moment gripping" can be measured reasonably well without asking a language model: loudness, pace, novelty, and a handful of phrasings carry a lot of the signal, and framing the trailer as an optimization (rising tension, fixed budget, forbidden region) produces cuts that feel deliberately edited. I also learned far more about ffmpeg than I planned to, and that the fastest way to debug a media pipeline is to probe the intermediate files before touching the code.
What's next for Cliffhang
Speaker-tracking crop so the talking head stays centered in the tight 9:16 format, a per-channel style profile (fonts, caption position, end card with "full video on YouTube"), and posting directly to Shorts and TikTok so the whole loop — long upload in, trailer live — is one step.
Log in or sign up for Devpost to join the conversation.