ClipForge

Clipforge Homepage
Light mode
Main page
Webhooks & Zapier Integration
Channel & Account
Comment Priority Classification
Video Clips with re-rendering option
Video Upload with flow

Inspiration

Every creator with a podcast, talk, or livestream sits on hours of raw footage that could be a dozen viral shorts — but turning it into that is 1–3 hours of grunt work per video: scrubbing for the good 30 seconds, cutting it to length, reframing to 9:16, burning in captions, writing a title/hashtags for three different platforms, and double-checking each platform's upload limits. That's the same repetitive pipeline every single time, which is exactly the kind of workload that should be one upload, not one afternoon. ClipForge is that upload.

What it does

Drop in a long-form video and ClipForge runs the entire repurposing pipeline for you:

Transcribes the full video with word-level timestamps (Groq Whisper)
Scores the opening hook's retention strength using audio signals + LLM analysis
Selects clip-worthy moments with an LLM grounded in the real transcript, then enforces duration/overlap rules deterministically
Trims dead air and filler words, renders to 9:16 with burned-in captions (three styles: block, minimal, karaoke), and loudness-normalizes the audio (EBU R128)
Checks every clip against current TikTok, Instagram Reels, and YouTube Shorts limits with actionable violation callouts
Generates 3 thumbnail variants, titles, descriptions, hashtags, SEO tags, and YouTube-style chapters
Repurposes the same footage into X/Twitter threads, LinkedIn posts, and newsletter sections
Scans for brand-safety risk (financial/medical/legal claims) and offers profanity bleeping
Exports .srt/.vtt/.csv and a one-click zip of everything

A batch queue processes multiple uploads sequentially with live position tracking at /queue, and a separate comment-moderation tool triages post-publish comments into ok/spam/toxic/needs_reply/off_topic.

How I built it

ClipForge is a pnpm monorepo with a Next.js 15 (App Router, React 19) web app and a separate Express/TypeScript worker, sharing a Postgres database (Supabase) and local storage. The browser polls job progress every 2 seconds during processing.

The pipeline itself is a chain of deterministic and LLM stages: Groq Whisper handles transcription (chunked and parallelized for long uploads, cached in Postgres by file hash), Gemini 2.5 Flash (with an OpenRouter fallback) handles clip selection, hook scoring, and metadata generation, and FFmpeg does the heavy lifting — libass caption burn-in, two-pass loudnorm for audio, and select/aselect filters for dead-air and filler-word trimming, all composed into a single filter pass per clip. Thumbnails are composited with sharp (real extracted frames + SVG gradient/text overlays). The worker enforces a concurrency cap of 1 so FFmpeg encodes and LLM calls never starve each other.

Challenges I ran into

Grounding LLM clip selection in reality: an LLM picking "clip-worthy moments" will happily hallucinate timestamps that don't land on sentence boundaries. I had to force selection to work directly against the transcript text and then deterministically snap and validate durations/overlaps afterward rather than trusting the model's numbers.
Making FFmpeg do five jobs in one pass: captions, dead-air/filler trimming, loudness normalization, and letterboxing all touch the same filter graph. Getting trimming to run before the loudnorm measure pass (so silence doesn't skew the loudness target) took some reordering.
Keeping the pipeline honest under concurrency: FFmpeg encodes and LLM calls both want CPU/network headroom, so multiple simultaneous jobs would starve each other. A sequential, single-concurrency worker queue with real-time position tracking turned out to be simpler and more reliable than trying to parallelize.
Platform compliance drift: TikTok/Reels/Shorts limits change and aren't always consistent across their own docs, so I had to source and verify current numbers and build the checks so they're easy to update later.

Accomplishments that I'm proud of

Getting the entire pipeline — transcription, hook scoring, clip selection, rendering, compliance checks, metadata, and thumbnails — to run end-to-end from a single upload with no manual steps in between, and having the batch queue survive one job failing without taking down the rest of the queue.

What I learned

How much of "AI video editing" is actually deterministic signal processing wearing an LLM's clothes — dead-air detection, loudness normalization, and platform compliance are all rule-based, and the LLM is only doing the two jobs a human editor's judgment is actually needed for: picking moments and writing copy.

What's next for ClipForge

Smart face tracking for dynamic 9:16 reframing (instead of static letterboxing), 1-click direct publishing via TikTok/Instagram/YouTube APIs, an in-browser waveform caption editor, speaker diarization for multi-color subtitles, and cloud object storage (S3/R2) so it's not tied to local disk.