Snap Chef

Gemini AI analyzing the image of a fridge with food.
The ingredients parsed by Gemini AI from the image.
Generated recipes matching the ingredients.
More ingredients information and instructions subtab.

Inspiration

With 40% of the entire US food supply ending up in the waste, helping to eliminate food waste—even at a small scale—can go a long way. We wanted to tackle the problem at a consumer level to help individual households use their leftovers more effectively. Furthermore, many people (our team included) spend parts of the day worrying about what they want to make for dinner with the ingredients in their fridge. We decided to chip away at both of these problems with a new app household cooks can use on a daily basis.

What it does

Snap chef just requires a picture of your fridge, either taken live or previously taken and uploaded. With this, it is able to use Google Gemini to parse the image and create delicious recipe ideas with the data and other parameters specified by the user.

Features

The app allows the user to manually edit the number and name of each ingredient in case Gemini incorrectly counted or some items were hidden. It has several sorting and filtering modes, so you can make sure you see recipes that use specified ingredients, or make sure you see recipes that meet your dietary restrictions. Lastly, you can also save recipes Snap Chef generated for later use.

App structure

The foundation of the app is built upon react native, developed with expo. It has a back-end server built with flask to pass user input to the Gemini Generative AI API.

Challenges

The free trial plan of Gemini we were using has a rate limit of 1 query every 5 minutes, which forced us to get creative with our development process and testing. The Gemini API servers were also having severe connection issues by the end of the final day, making testing and recording a test take over two hours. Our group barely had any react experience, so doing an entire react project and integrating it with other software was very challenging for us. In particular, getting the image picker and camera to work along with flask and the Gemini API took a lot of troubleshooting.

Accomplishments that we're proud of

Above all, we are happy to have competed in our first hackathon! Honestly, we're extremely proud that we managed to get the whole thing working and talking to each other and learning entire frameworks on the fly, given our lack of experience in this area. We are very proud of how far we've learned and grown from this project.

What we learned

Together, we all learned react-native, the Gemini GenerativeAI API, and flask for the first time! We also learned that it might not be the best idea to develop the frontend of a project before the backend.

What's next for Snap Chef

We hope to incorporate an actual recipe API, such as allrecipes or edamam, so we don't have to rely on Gemini for recipes. Furthermore, we hope to activate a paid plan for Gemini so we can get passed the 5 minute rate limit on requests. With these, it would be possible to put the app on the google play store and app store.