Sustainabite

Inspiration

OUR FOCUS Target user -> Students, struggling with food waste and budgets while adapting to cooking for themselves!

Main goal -> Provide a tool to reduce overconsumption, promoting usage of what you already have to minimize food waste.

What it does

Users are able to snap a quick image of their ingredients, pantry or refrigerator, fill out a few preferences on cuisine and cooking time and yield a few recipes to optimize the ingredients they have at home.

How we built it

We built a website based on a Kaggle dataset of 500,000 unique recipes.

IMAGE DETECTION -> Integrating CLIP by OpenAI, the model understands images and text in conjunction, enabling identification. -> Pulls ingredients from a list of 500+ common ingredients. -> Fine tuned on the common data base

OVERARCHING AI MODEL (code name: botboclaat) -> Trained on retrieving a recipe based on the following user inputs - Ingredients, scraped from image recognition - Cuisine, user input - Time spent for cooking, user input

CUISINE MODEL (code name: miss worldwide) -> Trained on predicting cuisine from ingredients list - American, Caribbean, Chinese, French, German, Greek, Indian, Irish, Italian, Japanese, Korean, Mexican, Moroccan, Spanish, Thai & Vietnamese

MEAL MODEL (code name: balerinna cappuncinna, mimimimi) -> Currently not in usage - When in usage minimized output a little too much -> Trained on predicting meal type based on ingredients list

TIME MODEL (code name: pwincesssss!) -> Trained on identifying time measurements based on tags

FRONTEND ON STREAMLIT -> Simple front end hosted through Streamlit -> User friendly and simple -> Key features include: - Find recipe; prompts user for input & locates recipe based on dataset - Pantry cupboard; saves items in pantry or places them onto grocery lists - Favorites; a collection of saved recipes

Challenges we ran into

A major challenge was nuances with working with such a large data base. With over 500,000 entries dating back to the 1990s, there were many discrepancies in syntax, inputs and tags. Although unexpected, this was not particularly difficult to deal with, just something to be mindful of.

Furthermore, a few of our models were not in the final product - our initial time predictor and the category model. The category model was simple in writing, but we realized it minimized our target recipes too much, so we removed it from the overarching model. However, the time predictor was much bigger of a hurdle. Although our initial dataset had time tags, they were not to the degree of precision we wanted to offer; simply stating "Under 60 minutes," etc. This led to acquiring a different data set to base time predictions off the summation of numerical figures and actionable steps within the instructions segment. However, after several hours and issues with running our main data set through the machine, we ended up settling for a multiple choice time value rather than a sliding scale.

Finally, as with all machine learning models, we spent countless hours fine tuning and training accuracy!

Accomplishments that we're proud of

We are super proud of our final product, as food waste is incredibly important to us. This project tackled a dilemma we face daily, as students trying to optimize what we have. Furthermore, we are proud of our various ML models as this is the first large scale model we have created together!

What we learned

We learned how to handle big data. Previously having worked with smaller sets, we were ready for this challenge, but we still found a learning curve. We definitely switched our focus to optimization to allow for productivity!

What's next for Sustainabite

With more refinement, we would also try to integrate diets such as plant-based, lactose-free, and celiac into our app as an option to toggle. We considered the addition of a chatbot to address specificities that our recommendation based model could not handle.

We would like to integrate more community focus. As students, our main social circles live in close proximity - something we'd like to take advantage of in our mission of eliminating food waste. A social component to our app would take Sustainabite to the next level - allowing for friends to share ingredients and meals together!