-
-
prefrences
-
main page
-
food detection
-
recipe 1
-
recipe 2
Inspiration
The urge of wanting to eat healthy while saving money, not wasting food, and having time and inspiration to cook is a real challenge most of us can probably relate to. Many people, including students and young workers, struggle a lot with maintaining a healthy diet and avoiding wasting money on overpriced food in this inflated economy. Since it's hard to find inspiration to cook with what we already have, groceries we've splurged on often go to waste too! This struggle is even more difficult for people with dietary restrictions, who face an additional barrier to meeting their specific dietary needs. As a result, they often end up not eating for long hours or resorting to lower-quality outside food. It is important to recognize that what we eat impacts our mood and mental health significantly due to the gut-brain axis. Thus, the goal is to reduce the struggle of cooking at home, optimize pantry/fridge contents, and find quick, healthy recipes that benefit our bodies while also reducing the financial pressure of buying outside food for anyone with any unique needs with the power of technology!
What it does
Nutrition pAIntry is a web app where you can select your dietary restrictions/preferences in your profile and by simply scanning your pantry or fridge you get healthy recipes based on the available food items. The embedded computer vision application captures the available items in your fridge or pantry, couples it with your preferences or restrictions, and guides you toward delicious, healthy diets that utilize the ingredients you already have. No need for extra shopping, thinking, or struggles to figure out what to eat or how to use groceries that are going bad, just scan what you have quickly and let the magic happen!
How we built it
The front-end application utilizes React.js and CSS. This involves creating the title page, the menu for dietary restrictions/preferences, setting up a page for the live webcam for computer vision, and creating pages to redirect users to appropriate recipes based on detected ingredients. We used the ml5 API, which is built from TensorFlow's built-in Coco-ssd model, to identify simple ingredients through computer vision. Additionally, all graphics on the website were created using Canva and Figma and later implemented on the front-end.
Challenges we ran into
Finding an algorithm suitable for detecting raw food items such as raw meat, rice, spaghetti, and other pantry items posed a challenge. There were few helpful databases available, as most food detection applications were focused on calorie counting and trained on prepared dishes. While we came across some OpenCV models that showed promise, they were outdated, and despite spending four hours debugging and trying different package versions, we couldn't achieve success.
Accomplishments that we're proud of
We are proud to have developed a fully functioning web app, despite working in a group of only two. Both of us had exams and homework assignments due on the day of the Hackathon but despite these challenges, we were able to create a project that we are extremely pleased with, particularly the frontend aesthetics and functionality.
What we learned
It was our first experience with computer vision and utilizing its outputs on the backend to implement functionality. Through this process, we gained valuable insights into how these algorithms function, the type of data they return, and how to interpret them effectively. Examining the data structures provided us with a better understanding of how machine learning models are constructed. We are now hopeful that we can apply this knowledge to develop our model for the full mobile app version of this project.
What's next for Nutrition pAIntary
Certainly, focusing on developing our own algorithm presents an exciting opportunity, especially considering it was a struggle to find any functional pantry/fridge item-detecting algorithm. By creating our own model, we could potentially offer innovative approaches to food detection that could benefit us and hopefully a wide audience. Moreover, transitioning the web app into a mobile app could extend its reach and ease of use, allowing the achievement of diverse dietary goals, saving money, and efficiently managing their pantry and fridge inventory.
Log in or sign up for Devpost to join the conversation.