For Princeton Hacks Spring of 2018, STDLIB challenged hackers to take their easy to use function API generator to create something amazing. We decided to take on one of our biggest gripes, finding recipes for images we find on social media sites like instagram.
What it does
The application is a google chrome extension that allows you to find a recipe of an image in your screen by simply clicking on the plugin button itself. The ingredients and recipe link are displayed below the plugin button.
How we built it
The application sends a screenshot of the current open tab to Clarifai which then analyses it using their public food model and returns the ingredients of the picture. A food recipe API is then queried from which we receive a recipe based on the ingredients Clarifai gave us given a certain threshold of probability. We then display the information to the user and send them text message containing the ingredients.
Challenges we ran into
The biggest issue was only working with 100 calls a day with Clarifai. We had to make 3 different accounts to test our code and then have enough calls to be able to demo it. The second biggest issue is the lack of free food recipes APIs, as the ones that are free are very limited and are more targeted towards giving social media friendly recipes.
Accomplishments that we're proud of
The application works fully, from getting the screenshot using the chrome extension library, to sending a text message through the STDLIB function.
What we learned
What's next for eatalytic
Get a better food API. Allow for the user to send feedback to the AI model. Being able to configure the phone to text. Being able to turn off the text feature. Create our own model.