ClipForger

Inspiration

Content creators spend hours manually finding highlights, trimming videos, adding captions, and converting content into vertical formats for social media. We wanted to build a tool that could automate this process and make short-form content creation accessible to everyone, especially small creators and students who may not have access to expensive editing software.

What it does

ClipForger transforms long-form videos into social-media-ready short clips. Users can upload videos or provide YouTube links, and the platform automatically transcribes speech, segments videos into clips, generates captions, and exports vertical videos optimized for TikTok, Instagram Reels, and YouTube Shorts. It also offers multiple rendering modes, including Standard and Cinematic experiences.

How we built it

We built ClipForger using FastAPI and Python for the backend and Next.js with TypeScript for the frontend. Groq Whisper Large V3 Turbo powers speech transcription and word-level timestamps. FFmpeg and NVIDIA NVENC handle video processing and GPU-accelerated rendering, while yt-dlp enables direct YouTube imports. Parallel processing significantly reduces rendering time and improves scalability.

Challenges we ran into

Video processing proved to be computationally expensive. We faced challenges with rendering performance, parallel processing, subtitle generation, and handling large video files efficiently. Designing rendering modes that work well for different types of content while keeping processing times low required several iterations and optimizations.

Accomplishments that we're proud of

We're proud of building a fully functional end-to-end AI video processing pipeline from scratch. The platform can automatically transform long videos into multiple short clips, generate captions, and render videos quickly using GPU acceleration and parallel processing. Achieving near real-time rendering performance on consumer hardware was a significant milestone.

What we learned

This project taught us much more than integrating AI APIs. We learned about building scalable processing pipelines, optimizing GPU workloads, managing asynchronous systems, designing responsive user experiences, and balancing performance with product features. We also gained hands-on experience with multimedia processing and concurrent programming.

What's next for ClipForger

Our next goals are to introduce Creator Mode with smart cropping and premium captions, improve highlight detection, provide real-time clip previews during rendering, and enhance the overall creator experience. We also plan to explore advanced AI features that can better understand video context and further reduce the manual effort required to create engaging short-form content.