Often when we watch videos or see a movie we remember seeing specific funny/interesting/sad moments that we would like to rewatch, but we don't want to watch the whole video to find where this is. This is why we created Framehunt, a platform where we can give the user specific tags of objects in the video, and where in the video these elements are.

What it does

Framehunt takes a YouTube video link, raw video link, or an uploaded video as input. In our backend, we call different API's to figure out the video's tags at specific frames. We then show the video and the most important tags to the user. The user can then search for a specific tag and we will output a button list of the timestamps where this tag appears on the video.

How we built it

We built the whole application using JavaScript. We used Node.js on the backend with Express and React + HTML + CSS on the frontend. We used the Clarifai API to help with object detection, and a few different tools to help with dealing with video files.

Challenges we ran into

We had a lot of difficulty dealing with video files and streams within the backend and sending them as requests. The requests were difficult due to being large files as opposed to just text.

Accomplishments that we're proud of

YouTube videos were especially difficult because of the video formatting but we were able to make them work eventually using a few NPM packages. We were also proud of being able to have a video player that seeks to specific points in the videos.

What we learned

We want to be able to handle full movies and have more expansive and accurate object recognition. Furthermore, we would like to be able to search in multiple videos, outputting the ones with better results first.

Share this project: