ClipSearch

Inspiration

As members of Cal TV, we've often found editing is one of the most time-consuming and frustrating stages of the post-production process.

When filming a video longer than a few minutes, we often run into several challenges.

An overwhelming amount of footage to sort through
Not enough man-power to get eyes on every video clip

For reference, a 10-minute Cal TV video uses on average 45 minutes of filmed content. That's at least 45 minutes of video to review - around 1 hour of sorting content into relevant topics - and 1 hour of editing the content into a finished package.

Post-production often takes almost three times as long as it takes to film the content itself, the case for many lower-budget film projects -- we hope to imagine a streamlined editing process guided by audio-based search functions.

What it does

ClipSearch converts daunting footage into digestible content. The website locates spoken keywords in uploaded footage to produce clips that feature the selected phrase.

Just upload and search to find relevant video clips in your footage ready to use!

How we built it

Our web app is built on Flask and HTML. When the user uploads a video, we process on our Flask backend, which converts to video to an audio file and submits it to an API for speech to text translation. Once the user searches for a word, we highlight it via native JavaScript and send the cutoff timestamps to our backend, which slices the video and returns it for rendering.

Challenges we ran into

One of our biggest challenges was finding a model that would accurately do speech to text translation. Initial models got most of the words wrong, but we found an API that was optimized for this translation. Other challenges included converting different video file codecs, and figuring out which portions of our system would be done on our backend vs frontend.

Accomplishments that we're proud of

In the span of 24 hours, we were able to build a working program that fits our specific media needs as a club, and became semi-proficient in new languages such as HTML and Python. While most of our team was made up of complete beginners to not only hackathons but coding itself, each of our teammates contributed to the project with either landing page design, coding, communications support, and always moral support.

What we learned

Speech-to-text models are harder to work with than we initially expected. We learned how to use Flask and how to implement basic API calls in Javascript.

What's next for ClipSearch

In an increasingly rapid short-form content media space, the need to quickly sort through video footage has never been greater. We hope to bring ClipSearch to other filmmakers, podcasters, and in particular TikTok creators/short content creators.