Snack-GPT

Inspiration

Did you know around 1/3 of all food produced globally is wasted every year, amounting to approximately 1.3 billion tons of food? This is enough to feed 3 billion people, yet nearly 1 in 9 people worldwide still face hunger. Food waste is a massive global issue, with millions of dollars worth of groceries thrown away every year. At the same time, people struggle to maintain a healthy diet and reach their weight goals. Most existing solutions focus on either meal planning or tracking calories, but none effectively combine nutrition, sustainability, and automation.

What it does

We present to you our solution, Snack-GPT! Our AI-powered Snack-GPT transforms the way users manage food and plan meals.

Seamless Pantry Management – Users can scan receipts to automatically log groceries, tracking ingredients, quantities, and expiration dates. They can also manually add, update, or remove items.

Personalized Meal Planning – Our system generates custom recipes based on pantry ingredients, prioritizing items that are expiring soon to reduce waste while ensuring users stay on track with their dietary goals.

Health & Fitness Integration – Users set their current weight, target weight, and time frame, and Snack-GPT suggests balanced meals that align with both their nutrition needs and available ingredients.

This is important because Snack-GPT reduces food waste by promoting sustainability, helps users eat healthier by aligning recipes with their diet plan, and saves time & money by maximizing pantry usage.

Imagine a world where you don’t waste food while effortlessly sticking to your diet. That’s the future Snack-GPT is creating.

How we built it

Our project was built using a combination of Streamlit for the frontend, Python for the backend, and Supabase for the database. The frontend was designed using Streamlit’s intuitive interface, which allowed us to quickly build an interactive and responsive application for users to scan receipts and manage pantry items. For the backend, we utilized Python to handle the logic behind user actions, including processing receipt images, interacting with the database, and generating recipes. Supabase served as our database solution, providing a seamless and scalable way to store user data, pantry items, and receipt details.

One of the key features of our project is the integration of EasyOCR, which we used for the first time to extract text from receipt images. This allowed us to automatically detect the ingredients listed in the scanned receipts. The extracted text is then processed and added as new rows in our Supabase database table, linking the items to the corresponding receipt. We also incorporated a recipe-generating feature using Gen AI, which automatically generates recipes based on the ingredients in the user’s pantry.

Together, these technologies worked in harmony to create a robust system that not only scans receipts but also helps users manage their pantry items and generate recipes with ease. The combination of Streamlit, Python, Supabase, EasyOCR, and Gen AI enabled us to build a powerful and dynamic application that offers real-time, intelligent features for users.

Challenges we ran into

One of the biggest challenges we encountered during the development of our app was that most of our team members had little to no experience working with Streamlit, which we chose as the framework for building our frontend. This posed a steep learning curve at first, as we had to quickly familiarize ourselves with how Streamlit operates, how to integrate it with the backend, and how to deploy the app effectively.

Another significant challenge was that it was our first time using EasyOCR for scanning and extracting text from receipt images. The process of training and fine-tuning EasyOCR to accurately detect and extract ingredients and other relevant information from receipts was new to us. We encountered some difficulties with accuracy and text formatting, which required adjustments to the OCR model and error handling techniques.

However, these challenges pushed us to dive deeper into optical character recognition (OCR) technology, and we eventually managed to get it working seamlessly. By leveraging EasyOCR, we added a powerful feature to our app, allowing users to scan receipts effortlessly and add items directly to their pantry.

Accomplishments that we're proud of

We are extremely proud of the progress we made with our project, particularly in the areas of receipt scanning and ingredient extraction. Using EasyOCR for the first time, we successfully built a system that scans receipts and automatically extracts ingredient names from the receipt images. This was a significant accomplishment, as it allowed us to seamlessly integrate OCR into our workflow, providing users with a powerful tool to digitize their groceries. Additionally, we practiced and experimented with Gen AI to create recipe suggestions, which was a new and exciting challenge for us.

One of our primary goals was to make our project unique, and we achieved this by focusing on a pantry management system. We wanted users to easily track and utilize the ingredients they have left in their fridge. By integrating recipe generation, we provided a solution that not only reduces food waste but also offers healthy, low-calorie recipe suggestions. This feature directly aligns with users' desire to stay healthy while achieving sustainability by managing food resources effectively. It’s rewarding to know that our project helps users manage their calories while offering practical and nutritious meal ideas.

What we learned

Throughout this project, we gained hands-on experience with EasyOCR for text extraction, which proved to be more effective than using the BERT model for our receipt scanning feature. We also learned to build a full-stack application using Streamlit, creating a dynamic, user-friendly interface. By integrating Supabase for database management, we improved our skills in backend development, specifically in handling data and relationships between tables. This project helped us enhance our understanding of OCR, frontend widgets, and backend database interactions, allowing us to create a smooth and functional app.

Additionally, we gained valuable experience working with Gen AI to generate recipe suggestions, providing a unique and personalized user experience. We also developed the ability to incorporate real-time data updates, allowing the app to automatically update users' pantry inventories based on the scanned receipt. This project pushed us to experiment with new technologies, deepen our problem-solving skills, and understand how to integrate multiple systems to create a cohesive, functional product. It was a great learning experience in both technical skills and project development.

What's next for Snack-GPT

Looking ahead, we have several exciting features in mind that will further enhance the user experience and make the app even more valuable. One idea is to take recipe suggestions a step further by recommending additional ingredients that users can add to their meals. These ingredients will not only enhance the recipe but will also introduce users to new, healthy meal options. To make this even more accessible, we plan to integrate a feature that suggests nearby stores where users can purchase the required ingredients, making it easier for them to find what they need.

Additionally, we are exploring the incorporation of voice commands into the app. By adding voice functionality, users will be able to speak the ingredients they have or want to add, and the app will automatically update their pantry or add new items based on their commands. This will make the experience more intuitive and hands-free, allowing users to interact with the app in a natural way.

These new features aim to improve accessibility, convenience, and usability, ensuring that users have a seamless and enjoyable experience while managing their pantry and making healthy meal choices. Join us in making sustainability and nutrition effortless!