Scrumptiousroll

Inspiration

We've been Internet friends for over 6 years, and two of us watched anime a lot. So much so, that they convinced the third to give it a try, and now we have anime nights whenever the stars of our schedules align.

Someone said "fries" while we were brainstorming an idea, and that was when Scrumptiousroll was born.

What it does

Scrumptiousroll is a Chrome extension that you can open while watching anime. If you see a delicious piece of food on screen, simply pause the video, open the extension, and press the "I see anime food!" button.

Scrumptiousroll will take a screenshot of your screen and process it to identify any food in the image. Then, it will redirect you to the top-rated recipe for that item from the Food Network.

All recipes are saved locally, so once you finished the episode, you can make all the anime dishes!

How we built it

We used a Chrome Extension, various custom scrapers to collect data, and a custom trained MobileNet model (we tried using Tensorflow but kept running into issues so we swapped over). All of the scriptings is written in JavaScript, though the model was written in Python before being imported to JavaScript.

Challenges we ran into

Models require a lot of data, and sometimes the formats aren't very nice. :c

Accomplishments that we're proud of

The title, the logo, and the fact that we taught one of our team members what carbonara is. Oh, and the machine learning and chrome extension were kind of cool too.

What we learned

Training our own model is harder than we thought (:

Images of animated food were rather difficult to find and classification was time consuming, so our data set for this prototype was only 120 images separated into 10 categories.