yt-auto-thumbnail

Inspiration

Every creator hits the same wall after finishing an edit: now you have to scrub back through the whole video looking for one decent frame, then jump into Photoshop or Canva to slap bold text over it. It's the same repetitive task on every single upload, and it's exactly the kind of "creator busywork" this hackathon was built around. I wanted to see how much of that pipeline I could automate end-to-end with a single command — no manual scrubbing, no design tool, no API costs.

What it does

Give it a raw video file, and it hands back a finished, ready-to-upload thumbnail:

1. Extracts candidate frames using scene-change detection, not just random sampling
2. Scores every candidate on sharpness, exposure/contrast, and whether there's a clear face in it, then ranks them
3. Transcribes the audio locally with Whisper and drafts a few short, punchy overlay-text options straight from what's actually said in the video
4. Composites the winning frame and winning text into a finished thumbnail

All in one command, and all running locally — no LLM API key required.

How I built it

I broke the pipeline into four independent scripts before wiring them together, which turned out to matter a lot for debugging:

• extract_frames.py uses OpenCV for frame extraction and histogram-based scene-change detection, with a periodic fallback so short clips still get enough candidates
• score_frames.py combines Laplacian-variance sharpness, brightness/ contrast heuristics, and Haar cascade face detection into a single ranking score
• transcribe_and_draft.py runs OpenAI Whisper locally for transcription, then scores transcript sentences with simple heuristics (ideal length, "power words," numbers, questions) to pick thumbnail-worthy phrases — deliberately avoiding an LLM API call to keep it free and offline
• overlay_text.py uses Pillow to composite auto-sized, stroke-outlined text onto the winning frame
• run_pipeline.py chains all four into one command for the actual demo experience

Challenges I ran into

• A brand-new OpenCV release broke on launch. opencv-python-headless 5.0.0 installed cleanly but was missing cv2.CascadeClassifier entirely — no conflicting packages, just a broken build. Pinning to <5 fixed it immediately, and I added that pin to requirements.txt so it doesn't bite anyone else running the setup.
• Short videos produced almost no candidate frames. My first version of the periodic-sampling fallback used a fixed 4-second interval, which meant a 12-second clip only yielded 5 candidates — not nearly enough to rank meaningfully. I rewrote it to scale the sampling interval to the video's own duration instead of a fixed constant.
• Real-world footage doesn't always have speech. A lot of short-form content is music-only or has no clear dialogue, and Whisper correctly returns an empty transcript for those. Instead of failing silently, I added a --manual-text fallback so the tool degrades gracefully instead of breaking on a very common case.
• Stale intermediate files silently corrupted later runs. Because candidates/ and ranked/ weren't cleared between runs, old frames from an earlier test video were still being scored alongside a completely different video's frames. Fixed by clearing both directories at the start of each run.

What I learned

The scoring and compositing logic was the "fun" part, but almost all the real engineering time went into making the pipeline robust to messy, real-world input — a brand-new dependency version with a broken build, a video too short for my sampling assumptions, and footage with no speech at all. Building it as small, independently testable scripts before wiring them into run_pipeline.py made every one of those bugs much faster to isolate and fix than if I'd written it as one big script from the start.

Built With

• ffmpeg
• git
• github
• numpy
• openai
• opencv
• pillow
• python