ESPMT

Inspiration

Look at this fragment of the movie "The Secret Life of Walter Mitty (2013)": https://www.youtube.com/watch?v=kurY9aSolo0

Because the outdoor views are so amazing, the only questions we kinda ask ourselves while watching this fragment: when am I going to Iceland?

If at that moment you received any sort of notification "Go to this website and book your trip to Iceland. Your personal discount is 30%.", the chance is high - you would go there.

What it does

See Inspiration. This is the concept that we aimed to prototype.

How I built it

We have not.

Challenges I ran into

First, we thought that the use of public image recognition services will be enough to identify "matches" between videos and travel agency propositions. We tried to use "Eiffel tower" as an example. We submitted a number of Eiffel tower images to AWS Recognizion, and it responded with the following concepts: "architecture, building, outdoor, dark, light". That's how we realized that we need to train our own model.

Then we tried to use Tensorflow to classify images. We uploaded about 400 images of Paris (with Eifel tower), and about 400 images with traditional Amsterdam buildings. While testing the model, we managed to have a very good recognition of Eifel tower, however, test against Copenhagen photo was a failure (because it recommended Paris).

The next step to enrich the model would be:

• for each proposition of travel agency: extract by basic keywords the images from Google Images search and use them to train the model (for extraction of such keywords from the content we planned to use Text recognition API (to extract keywords from travel agency proposition, like Paris, Amsterdam etc). These images would be used for training our model
• then we would use video prediction API to perform prediction, and based on it - making propositions to the watcher at the end of the movie.

See https://www.clarifai.com/developer/guide/predict#videos (we planned to use it for training the model, and making the prediction using this model).

Accomplishments that I'm proud of + What I learned

This is our first experience with Tensorflow. Moreover, we don't have experience with AI, and diving into the concepts was eyeopening.

Being proud leads to self-confidence, so we decided to not focus on that.

What's next for ESPMT

Dunno, sorry. Too much, one minute till the end of submission deadline :-P