NutriSnap: Eat Smart, Live Better

Inspiration 😊

Obesity, diabetes, hypertension, and cardiovascular disease are among the top health threats in the US and many other parts of the world. One of the main reasons behind this is unwise food choices. With better insights into the food we purchase and healthier alternatives, we can make more informed decisions, leading to a healthier and happier lifestyle.

What it does 🛠️📲

The app allows users to capture a picture of a food label. It extracts key nutritional information from the label, including total_fat, saturated_fat, sugars, protein, sodium, and calories. Based on the food category, if the item is deemed an unhealthy choice, the app provides healthier alternatives using a database of about 9,000 food entries.

How we built it 🏗️🤖

The app includes:

Frontend: Built using Streamlit, it captures images, processes them, and presents suggestions along with the extracted food ingredients and healthier alternatives.
Backend: Images are stored in AWS S3, and the GPT-4o mini API is triggered to process food labels and categorize food.
ML model: A recommendation system is built using a structured dataset containing approximately 9,000 food items and their nutritional facts. The food label information is used to filter these entries to find healthier options within the same category.

Challenges we ran into 🚧🤯

Lack of testing data: Collecting food labels was challenging. Our team and friends contributed by taking photos of food labels from their fridges or pantries, compiling them into a dataset.
Cost of the app: Costs arose from maintaining the backend server, frontend, and API usage. We explored AWS, Railway, Render, and Streamlit Cloud. Eventually, we selected AWS S3 and Railway for the backend and database, Render for the frontend, and OpenAI GPT-4o-mini, which perfectly met our needs.
Late entry and steep learning curves: We joined the hackathon late and lacked full-stack expertise. We had to learn quickly and build the project on the fly.

Accomplishments that we're proud of 🏅🎉

Successfully building a working app within a short timeframe.
Collecting over 100 food label photos from our kitchens.
Compiling a food ingredient database for the recommendation system.
Testing the app with friends and family, who provided valuable feedback and suggestions.

What we learned 📚🧠

Full-stack app development skills.
Selecting infrastructure that fits our budget and needs.
The importance of CICD (Continuous Integration and Continuous Deployment) for developing a web app.
Handling real-life datasets, which can be noisy with blurry photos, different orientations, or even photos of pets instead of food. We learned to manage these challenges in our data ETL and machine learning pipeline to have a robust ML pipeline.