Recipics

• 

  User GUI for taking a picture of their fridge/pantry or uploading an image

• 

  Users are able to either delete ingredients that the algorithm identified using the trash can icon or add ingredients of their volition

• 

  A list of possible recipes given the user's ingredients whose hyperlinks can be clicked on

• 

  Sample image of the user selecting multiple pictures of ingredients to process

• 

  The submit button appears after the user uploads all of the ingredient images they want

• 

  A sample classification of an image containing ingredients for a slow-cooked chicken stew by our CNN model

Inspiration

Our goal is to provide viable recipes for cooks who are just starting out--especially fellow college students who may be in the same boat. In addition, we want to contribute to the overall quality of life in our community by reducing food waste and ensuring quality nutrition.

What it does

Recipics provides the user with recipes that can be made with only a set number of ingredients. The end goal of the app is to let the user take a picture of the ingredients, and the app will recognize the ingredients that the user has and allow the user to add or remove some ingredients if they wish. We look at your preferences to cater to your health goals.

How we built it

We built this app to generate user-specific recipes from images of ingredients. We developed a Convolutional Neural Network (CNN) using YOLOv8 for efficient ingredient detection, achieving an 83.7% precision score. We created a custom dataset of over 300 labeled photos of raw ingredients, split into training (89%), validation (7%), and testing (4%) sets.

For recipe recommendations, we compiled a database of over 30,000 unique recipes using MongoDB, enhancing search speed with 10 key parameters. Our model is hosted on Roboflow to optimize load times on our Flask server.

Challenges we ran into

1. Aggregating a custom dataset was time-consuming, necessitating supervised learning due to limited images.
2. Our mapping algorithm sometimes yielded inconsistent results, affecting ingredient accuracy.
3. The AI model occasionally misidentified input images, leading to questionable accuracy levels.

Accomplishments that we're proud of

1. Rick developed a Flask backend to interpret receipt images, which filtered and mapped text data onto a dataset of raw ingredients.
2. Jason designed the mobile app's UI/UX using React Native and integrated frontend and backend features for enhanced filtering options.
3. Dhruv specialized in CNN model creation and training with Roboflow.
4. Dev automated the insertion of 30,000+ recipes into the database and created a string matching algorithm for efficient data retrieval.

What we learned

• Gained insights into Git, dataset quality, and model accuracy.
• Learned to process images for ingredient extraction.
• Gained knowledge on supervised vs. unsupervised learning.
• Developed skills in MongoDB database engineering, server deployment, and natural language querying.
• Learned to create virtual environments and integrate APIs and databases effectively.

What's next for Recipics

As we take Recipics to greater heights, our goals include expanding the size of our recipe database, fine-tuning our model's precision and recognition of obscure ingredients, improving runtime efficiency, and deploying to AWS.

Built With

• bson
• cv2
• flask
• fuzzywuzzy
• image
• imagetotext
• inference-sdk
• jsonify
• mongoclient
• mongodb
• numpy
• os
• output
• pytesseract
• python
• re
• react-native
• request
• requests
• yolov8