ClipWise

Smartly auto-clips the most engaging moments from YouTube videos using transcripts and AI — fast, serverless, and API-first.

Comment

Inspiration

Watching long videos to find short, impactful moments was frustrating — especially for educational or trending content. I wanted a tool that could automatically detect and extract the most engaging 30-second clips from any YouTube video using only its transcript. That’s where ClipWise started.

What it does ClipWise takes a YouTube URL, pulls the transcript, detects high-engagement segments using heuristics or LLMs(in feature realse v2), downloads the video, clips the most interesting 30-second portions, and returns them as pre-signed S3 URLs.

How I built it

  • Transcript is fetched using youtube-transcript-api
  • Logic identifies 2–3 "clip-worthy" segments (each under 30s)
  • Video is downloaded in AWS Lambda using yt-dlp
  • Clips are created using moviepy in Lambda’s /tmp
  • Final clips are stored in S3 and shared via signed URLs
  • Metadata is tracked in DynamoDB
  • Entire backend is serverless, API-first, and modular

Challenges I ran into

  • YouTube bot detection blocked downloads frequently
  • yt-dlp required cookie and anti-scraping workarounds
  • Lambda’s 512MB temp storage limit forced aggressive cleanup
  • Debugging headless code with no frontend required creative logging

Accomplishments that I'm proud of -Built and deployed a fully serverless video clipping pipeline — live in production, returning signed clip URLs with minimal latency.

-Implemented smart caching and reuse logic — when I first linked the YouTube video ("Blue - One Love"), clip generation took time as the content wasn’t cached. On subsequent requests, the clip loaded in just 2–3 seconds because my system detected the existing result in S3 and skipped redundant processing.

-Designed with modularity for the future — the architecture is clean and extensible, ready to support monetization, user authentication, and analytics.

What I learned

  • Practical use of yt-dlp, moviepy, and Lambda optimization
  • Handling API rate limits and YouTube scraping limitations
  • Designing clean APIs for a multi-step ML-driven workflow
  • How to make a CLI-first tool behave production-ready

What's next for ClipWisw

  • Add user accounts with persistent clip history
  • Support longer videos and multilingual transcripts
  • Use amazon bedrock or Train custom ML models to auto-score clip "engagement"
  • Build a polished web frontend
  • Add monetization and auto-publishing to YouTube Shorts/Instagram

Built With

Share this project:

Updates