Recipe Me!

Application Overview
Adding Ingredients
Filtering System
Saved/Favorited Recipes and Visual Pantry
Custom Computer Vision API Process
Precision vs. Recall API Stats

Inspiration

The dilemma of what to eat rises in every instance and piles stress upon an already busy day. You know you just went on a grocery run, but it feels as if you have nothing at all to cook. These simple predicaments can be resolved with a tool that consolidates the contents of your fridge, pantry, and any other food lying around. While there are ways of approaching this such as writing everything in a journal, we believe that creating an accessible tool with the least amount work would best address these dilemmas.

What's Recipe Me!

This mobile application provides a clear and organized way to approach meal planning by reducing the time spent deciding what to cook. Simply point the camera towards a food item to categorize it in real time and catalog the scanned food ingredients into a virtual pantry. The user can then receive catered recipes and track the food they possess, which can be applied to meal planning, expiration date tracking, and simplify many more processes. In addition, the application suggests a wide variety of recipes that are presented if almost all the ingredients are in the pantry, allowing the user to plan for future meals by adding ingredients to a shopping list. To further cater this feature to the user's interests, filters allow the recipes displayed to be based on a certain dietary restriction or meal specifications. For example, if a certain ingredient is going to expire soon, recipes using it will be prioritized to be sent to the user. Through this, we hope the users are able to reduce the amount of expired food thrown out and also have an efficient tool to base all their cooking decisions off of.

How we Built it

Utilizing the Microsoft Azure Custom Computer Vision Service, we were able to train the API for the ingredients in the recipes the user could be provided with. In order to provide whole and accurate training, we made sure to include as many forms of the ingredient, allowing the user to not be restricted to a singular presentation of an ingredient to have it be recognized. Furthermore, we mainly used Java to determine which recipes would be provided to the user based on their pantry's contents. While we did consider more advanced search algorithms to determine the presence of certain ingredients in the pantry, we selected to pursue with the simpler and more intuitive approach.

Challenges we ran into

Accustoming ourselves to the Custom Vision API and understanding the factors that go into training the service was our first challenge because we were not receiving the prediction and recall rates that we desired. However, we realized that we need to train using more images and a wider variety of the same ingredient, so the API would be extremely specialized to the specific tags, or ingredients, we trained. In addition, with it being our first time working with this API, we initially struggled with its integration into the Android application. Laying out the logic and determining how Recipe Me! and the API would connect provided a clearer picture, which led us to eventually figure it out.

Accomplishments that we're proud of

In addition to successfully integrating the API into the Android application, we believe our project reaches a larger audience with a more unconventional usage of our application. Along with meal planning and ingredient tracking, the camera detection software, recipe suggestions, and shopping list features double as accessibility features. These could help the blind and mute communicate their needs to others, specifically at the grocery store, where adding text-to-speech capabilities for the recipe suggestions and shopping list could further their day-to-day experiences.

What we've learned

We have gained experience with applying our coding skills to real-world issues and also learned how to custom train an API.

What's next for Recipe Me!

We hope to fully add in the filtering system to allow recipes to be catered to users beyond just the pantry contents. Adding this functionality would help those who face dietary restrictions and often find it difficult to find recipes that are specific to their needs. With the implementation of more catered recipes, we also hope to create a feature on the landing page that has the saved or favorited recipes on top for easy access.