Often when we watch videos or see a movie we remember seeing specific funny/interesting/sad moments that we would like to rewatch, but we don't want to watch the whole video to find where this is. This is why we created Framehunt, a platform where we can give the user specific tags of objects in the video, and where in the video these elements are.
What it does
Framehunt takes a YouTube video link, raw video link, or an uploaded video as input. In our backend, we call different API's to figure out the video's tags at specific frames. We then show the video and the most important tags to the user. The user can then search for a specific tag and we will output a button list of the timestamps where this tag appears on the video.
How we built it
Challenges we ran into
We had a lot of difficulty dealing with video files and streams within the backend and sending them as requests. The requests were difficult due to being large files as opposed to just text.
Accomplishments that we're proud of
YouTube videos were especially difficult because of the video formatting but we were able to make them work eventually using a few NPM packages. We were also proud of being able to have a video player that seeks to specific points in the videos.
What we learned
We want to be able to handle full movies and have more expansive and accurate object recognition. Furthermore, we would like to be able to search in multiple videos, outputting the ones with better results first.