Main page with clip suggestions
Full Video Editor

Inspiration

Clipping is a huge industry and timing matters a lot. The algorithm tends to reward whoever posts a moment first, which means creators need to find and publish clips while the stream is still live, not hours after it ends.

I also wanted to build something I would personally use. I stream for hours at a time, and I wanted a tool that lets me stay focused on the stream while Clippy quietly finds and prepares the moments worth sharing.

What it does

Clippy turns livestreams into clips in near real time. It captures YouTube and Twitch streams, generates transcripts, finds the strong moments, ranks the clip candidates, and renders everything locally on your machine.

It also handles editing, captions, and AI-powered reframing for different formats. Your source media and finished clips stay on your device by default.

How we built it

Clippy is a Tauri desktop app built with React and Rust. The Rust worker handles capture, transcription, AI analysis, rendering, and recovery, with SQLite and FFmpeg doing a lot of the heavy lifting.

A Cloudflare control plane manages authentication, device pairing, credits, and metadata sync. We use OpenAI's Responses API to analyze video frames and recommend layouts for different aspect ratios, and every AI response gets validated against structured schemas before we render a single clip.

Challenges we ran into

The hardest part was making the pipeline reliable while the stream is still happening. Interrupted segments, overlapping transcripts, duplicate clip candidates, app restarts, network failures, malformed AI responses, we had to handle all of it gracefully and keep the pipeline moving.

We also had to balance cloud inference with privacy, so media and full transcripts stay local whenever possible.

Accomplishments that we're proud of

We built a complete workflow from live capture all the way to a locally rendered clip. Clippy can recover from restarts, validate AI-generated candidates, render verified video, and give you a real workspace for reviewing and editing your clips.

We're especially proud that we designed it around what live creators actually need. Most tools hand you timestamps after the stream ends, and by then the moment has already passed.

What we learned

Building a reliable AI product takes a lot more than a model call. The systems around the model, the validation, the state management, the recovery, the editing, the rendering, all of that matters just as much as the AI itself.

We also learned that a local-first architecture is a win for both privacy and the actual user experience.

What's next for Clippy

Next up is improving live-edge speed, captions, vertical reframing, platform-specific exports, and one-click publishing to Shorts, TikTok, and Reels.

The long-term vision is for Clippy to become a real-time editing layer for livestreams, finding the good moments and having them ready before the stream is even over.