Common movie recommendation systems propose the content based on restricted characteristics - users that watch comedies are advised comedies and users interested in superheroes are advised DC movies. This makes the decision process extremely long and laborious. We want to bring the content discovery to the new level by using content analysis.

What it does

We have developed a user client and an application for TV.

The user opens an application on his smartphone and sees a set of movie scenes. He scrolls through them and selects the one he likes. The algorithm analyzes the video contents in real-time and generates the suitable keywords. For example, if the user stopped at the underwater scene of James Bond movie, he might follow the "underwater" keyword and discover movies with similar underwater scenes. After finding the video he likes, the user can simply rotate the smartphone and start watching the movie. Or, the user can just swipe the scene and save it for later. The system will remember the preference of the user and take it into account in further recommendations. The movie and the specific scene will also be synchronized with the TV application - the user can choose a movie in the bus on his way home and start watching it straight after he gets there.

How we built it

We built an iOS application with Swift and prototyping tools like Sketch. Video analysis is done by analyzing images on per-frame basis and this is done Clarifai API. In future, more accuracy can be added by analyzing the scripts and dialogs. The TV app was built with React. The movie backend is hosted on Heroku.

Accomplishments that we're proud of

Smooth and engaging user experience and accurate keyword selection.

Share this project: