GroceryTrak

YOLO Model
Flutter App

Inspiration

Research shows that a person's healthcare outcomes are directly influenced by their diet, making nutritional choices a critical factor in overall well-being. GroceryTrak was born out of the need for a smarter way to manage ingredients and recipes, ensuring users can make informed dietary choices. With growing concerns around food intolerances and diet restrictions, we aimed to create a seamless experience that helps users track groceries, find suitable recipes, and detect ingredients efficiently.

What it does

GroceryTrak allows users to:

Scan and identify groceries using an AI-powered image recognition system.
Manage their inventory of ingredients.
Find recipes that match available ingredients and dietary preferences.
Filter recipes based on intolerances and dietary restrictions.
Store and track ingredients to minimize food waste.

How we built it

The mobile application is built using Flutter, providing a responsive and intuitive user interface. The backend is developed in Golang and PostgreSQL, ensuring high performance and scalability. We use YOLO for real-time grocery detection. The dataset was compiled from publicly available sources on Roboflow Universe, merging multiple datasets to create a robust model that can detect a diverse variety of grocery items. The final dataset contains 62,000 training images, 12,000 validation images, and 7,000 test images across 503 classes of groceries. After training for 86,400 seconds, our model achieved an accuracy of 91%. We host the application on AWS EC2 with additional services from Google Cloud Platform (GCP) (for domain registration) and Hugging Face (for YOLO model hosting).

Challenges we ran into

Finding a well-labeled dataset for grocery detection was a significant challenge. To overcome this, we leveraged pre-trained YOLO models and fine-tuned them with additional data. Combining the Golang backend, Flutter frontend, and YOLO-based image recognition seamlessly required rigorous debugging and optimization.

Accomplishments that we're proud of

We successfully built an end-to-end grocery tracking system with AI-powered ingredient detection. We integrated recipe recommendations based on available ingredients and dietary restrictions. The entire system was deployed on AWS EC2.

What we learned

We realized the importance of high-quality, diverse datasets for training an accurate YOLO model far too late after we had spent half of our time training the first version of the model.

What's next for GroceryTrak

We found that our model still contains inaccuracies. We plan to find better data in the future to enhance the model’s capabilities. We will probably add data for recipes and ingredients from USDA’s database of spoonacular’s API to cater to a wider range of dietary needs. On the front end, we plan to add a feature where users can annotate detected groceries to improve model training.