No captions vs Captions
The Studio
Uploading to Youtube

Inspiration

A few months ago, my younger brother decided to try out for the student council. To try out, he had to make a short video explaining why he was the best candidate for the role. After days of work, he finally finished his video 2 hours before the deadline. While reviewing it one last time, something felt odd. That's when he realized he forgot to put captions, one of the most, if not, the most important parts of a video. He was already burned out, so, like any good brother, I offered to help him. After spending far too much time adding styled, synced captions, I finally finished, with only three minutes to spare. When I first thought of what would help a content creator, this exact situation came straight to mind, and that's the whole reason I built Kaptra, to help myself, my brother, and everyone else in the same boat as I was.

What it does

Drop in a clip, and Kaptra transcribes it with Whisper down to each word, so every caption syncs perfectly. While that's happening, Claude reads the transcript and notes down the words that matter most, where an emphasis color will be placed for a powerful effect. After that, either you can style yourself, or use our own styling feature to recommend the best possibly styled captions you can. Finally, you're just a few clicks away from downloading and uploading to youtube.

How we built it

The frontend is Next.js and TypeScript, while the backend is a small FastAPI service. The captioning uses Faster-Whisper to increase the speed of the process rather than using regular Whisper. All of the styling is in the front end, and the backend just runs one ffmpeg to burn the captions in, so what you see in the preview is exactly what you end up with.

Challenges we ran into

To start, at first I only used regular Whisper. It worked, but not nearly as fast as I wanted. After researching for an embarrassingly long time, I stumbled across faster-whisper, which completely enhanced the entire website, making everything so much faster with no change in accuracy. However, another problem arose, uploading to youtube. No matter what I tried, I just couldn’t

After that, yet another issue occurred. While making everything public, the captions suddenly stopped syncing. I didn't understand why, nothing was visibly wrong, until I realized only half the site was actually online. Vercel was hosting the front end, but Whisper and ffmpeg run in a Python backend that wasn't deployed, so the site was using a sample transcript and putting it over whatever clip I dropped in. The words were so similar to the video I was using that it took me way too long to notice.

Accomplishments that we're proud of

Honestly, after troubleshooting for so long, seeing the captions first pop up and actually work was a huge motivation booster for me. Seeing a tool that I would kill for months ago actually come to life is a great feeling, one that was only possible because of the effort I put in. Also, figuring out how to have captions to music felt so cool. It was like I could make my own music videos with just a clip of audio.

What we learned

I learned that transcription isn’t actually the hardest part. Surprisingly, it was all the, not smaller, but definitely more overlooked parts. Different settings between music and speech, making the captions sync, even just thinking of the countless customization settings, are just a few examples. Also,

What's next for Kaptra

The next step for Kaptra is to host the backend so the website runs live, and get the google project verified so uploads can go public. After that’s done, I plan to make Kaptra work for longform videos also, and videos of all sizes, so this tool can be used for anyone, anywhere.