view fAInder

Shop what you find interesting on video!

Comment

Inspiration

How many times have you wondered what the actors on a video are wearing? Or which cool city it is being shot on? With view fAInder you have an intelligent agent that knows what is interesting for you & guides you to websites where you can find more information & buy them.

What it does

When you watch any video on Maxdome and find something interesting on screen, press a button & you get all the interesting products like the attire of the cast, cities it is being shot in along with the weather in the location. The links can direct you to the websites selling the products like tour packages(weg.de) or attires(stylight).

How we built it

We built a Chrome plugin that is used to analyze the video content on Maxdome. The interesting parts of the video are sent to a machine learning model that detects and classifies the products of interest like locations & apparel. The interesting products are then fetched from the API partners like weg.de, wetter.com & stylight & presented to the user on the same screen. The user can go to the products directly from the Maxdome player interface.

Challenges we ran into

The inbuilt Computer Vision models provided by the Azure Cognitive Services were not that great for identifying products for our use case. So, we trained the model using the Custom Computer Vision service that needs a lot of images for training. Some of the API services like stylight.de were not available all the time. And weg.de API could be improved as it lacks some of the features like direct links which would have been handy.

Accomplishments that we're proud of

We managed to get the complete product across different platforms functional in the short amount of time.

What we learned

Always test your most important assumptions as early as possible. This helps avoid last minute surprises.

What's next for view fAInder

Improve the Machine Learning model may be using our own services instead of the Azure services. Deep learning could be used to built the custom model. It can also be implemented for other Video platforms like YouTube, Netflix, Amazon Video, etc.

Built With

Share this project:

Updates