Format Fluid

Interface of How Videos will convert
2 steps complete
Landing Page
Different Features
Types of caption styles
Demo Mode Part 1
Demo Mode Part 2

The Problem

Every creator knows the Sunday night dread. You spend all day making a great YouTube video. Then you remember: you still need 5 TikToks, 3 Shorts, a Reel, and a Twitter thread. That's 4-6 hours of repetitive busywork — cutting, reframing, captioning, rewriting copy for each platform.

I kept seeing creators burn out not from making content, but from repurposing it.

What I Built

Format Fluid is a single-page web app that automates the entire repurposing pipeline:

Upload any long-form video
AI finds viral moments using audio energy (EBU R128 loudness), face detection, and speech density
Auto-reframes to 9:16 vertical with smart face-centered cropping
Burns captions word-synced directly into the video via FFmpeg drawtext
Generates platform-native copy — different hooks, tones, and hashtags for TikTok, YouTube Shorts, Instagram Reels, X/Twitter, and LinkedIn

How I Built It

Python + Flask for the backend
OpenAI Whisper (base) for transcription with word-level timestamps
OpenCV for face detection and smart crop centering
FFmpeg for everything video — extraction, vertical reframing, caption burning
Vanilla JS frontend, no frameworks, just a clean drag-drop UI

The scoring algorithm combines three signals:

40% audio energy — loud, dynamic moments = engaging
30% face presence — close-up face shots perform better on short-form
30% word density — rapid speech = information-rich segments

Challenges

The biggest headache was OpenCV's face detection on macOS. cv2.CascadeClassifier kept breaking with different opencv-python versions. I ended up writing a graceful fallback — if face detection fails, the app weights audio energy + speech density at 50/50 and still finds great clips. It actually made the pipeline more robust.

FFmpeg's drawtext filter was another battle. Escaping special characters for shell commands while keeping word-sync timing accurate took way longer than expected. I chunked captions into ~3-second blocks for readability instead of word-by-word, which turned out cleaner visually.

What I Learned

Whisper's word_timestamps=True is incredibly precise — way better than I expected for a "base" model
FFmpeg is underrated as a video processing engine. Most people reach for Python video libraries, but raw FFmpeg commands are faster and more reliable
The best hackathon features are the ones you cut. I dropped a "thumbnail generator" idea halfway through because it diluted the core demo. Scope murder wins.

Why This Matters

Most automation tools do one thing. Format Fluid does the entire handoff — the part where creators lose 40% of their week to context switching. No API keys. No cloud bills. Runs entirely on your machine.