"Hey guys!" said Grant. "See these beautiful horses in this video?"
Jeremy dropped his XBox controller and slowly turned his gaze towards Grant's monitor. "I don't see anything?" he queried.
Grant sighed, "Well, it's too late now, the video has moved on."
Doug piped up. "Wouldn't it be cool if we could somehow see the timestamps of interesting objects in YouTube videos?"
"Well, we actually can, using machine learning!" suggested Alex.
And thus, an idea was born.
What it does
Upon navigating to the website, users can enter a YouTube URL to analyze, or select from a database of already cached videos. After entering a URL, users must wait for the website to process the video. Upon completion, the page returns a list of the top ten most relevant tags, along with click-able timestamps to find the items.
How we built it
We used TensorFlow and ImageNet for the classification and modeling of the neural network. We used Flask and Amazon Web Services (AWS) for the web application, and pytube and openCV for the video processing.
Challenges we ran into
It took a significant time to set up the web server.
Accomplishments that we're proud of
The wide scope of areas this project covered, ranging from low level machine learning algorithms to front end web development.
What we learned
We learned about how to set up a server on AWS, and how to use machine learning algorithms to classify various types of data.
What's next for TensorTube
We plan on parallelizing the object recognition in order to give an immense speed up in processing and compute time.