For Princeton Hacks Spring of 2018, STDLIB challenged hackers to take their easy to use function API generator to create something amazing. We decided to take on one of our biggest gripes, finding recipes for images we find on social media sites like instagram.

What it does

The application is a google chrome extension that allows you to find a recipe of an image in your screen by simply clicking on the plugin button itself. The ingredients and recipe link are displayed below the plugin button.

How we built it

The application sends a screenshot of the current open tab to Clarifai which then analyses it using their public food model and returns the ingredients of the picture. A food recipe API is then queried from which we receive a recipe based on the ingredients Clarifai gave us given a certain threshold of probability. We then display the information to the user and send them text message containing the ingredients.

Challenges we ran into

The biggest issue was only working with 100 calls a day with Clarifai. We had to make 3 different accounts to test our code and then have enough calls to be able to demo it. The second biggest issue is the lack of free food recipes APIs, as the ones that are free are very limited and are more targeted towards giving social media friendly recipes.

Accomplishments that we're proud of

The application works fully, from getting the screenshot using the chrome extension library, to sending a text message through the STDLIB function.

What we learned

Our team was not very familiar with javascript, but thanks to the great google and STDLIB documentation we were able to learn how to use it. This was also a great opportunity for us because we learned the basic concepts of machine learning by working with the Clarifai API. As a backend developer, it was also very useful to learn how to use STDLIB as now I don't have to worry about setting up infrastructure, and focus more on the code itself.

What's next for eatalytic

Get a better food API. Allow for the user to send feedback to the AI model. Being able to configure the phone to text. Being able to turn off the text feature. Create our own model.

Share this project: