Framehunt

Objects within video frames
Searching for specific tags
Options to search

Inspiration

Often when we watch videos or see a movie we remember seeing specific funny/interesting/sad moments that we would like to rewatch, but we don't want to watch the whole video to find where this is. This is why we created Framehunt, a platform where we can give the user specific tags of objects in the video, and where in the video these elements are.

What it does

Framehunt takes a YouTube video link, raw video link, or an uploaded video as input. In our backend, we call different API's to figure out the video's tags at specific frames. We then show the video and the most important tags to the user. The user can then search for a specific tag and we will output a button list of the timestamps where this tag appears on the video.

How we built it

We built the whole application using JavaScript. We used Node.js on the backend with Express and React + HTML + CSS on the frontend. We used the Clarifai API to help with object detection, and a few different tools to help with dealing with video files.

Challenges we ran into

We had a lot of difficulty dealing with video files and streams within the backend and sending them as requests. The requests were difficult due to being large files as opposed to just text.

Accomplishments that we're proud of

YouTube videos were especially difficult because of the video formatting but we were able to make them work eventually using a few NPM packages. We were also proud of being able to have a video player that seeks to specific points in the videos.

What we learned

We want to be able to handle full movies and have more expansive and accurate object recognition. Furthermore, we would like to be able to search in multiple videos, outputting the ones with better results first.

Built With

Submitted to

HackGT 2017
- Winner Top 8
- Winner Top 3

Created by

I worked on creating the Node.js + Express backend (API) to handle client requests and business logic. I also configured the AWS EC2 instance and worked on some design elements.

Thomas Thachil
I worked on setting up the Clarifai (external API) queries and on the file conversion tasks. I also worked on the network infrastructure and communication between components (react and express servers between one another)

TommasoPieroncini
I worked on the Clarifai API integration, data flow, and state management of the web application. I also built all the Front End with ReactJS and oversaw the design of the application.

Aayush Kumar
aayush.kumar@gatech.edu
I worked on the react web application handling the video file upload and flow in the components.

AlanisDavid