NutriFlow AI

NutriFlow AI is your Pocket Dietitian. It uses a fridge camera and AI to see what you have, understand your health needs, and instantly suggest meal plans.

Comment

Inspiration

Coming into Code4Hope, our team analyzed every company profile one by one using a feasibility analysis. After going through each problem's relevance, opportunities for innovation, and practicality, we chose an issue that stood out: Healthy eating. Because in the chaos of daily student life, classes, extracurriculars, and student responsibilities, we tend to forget to eat healthily. Thus, we choose NutriFlow, a company dedicated to helping people live and eat healthier.

What it does

Introducing the Pocket Dietitian: a dynamic meal-planner app Our app utilizes machine learning and a fridge camera to track food, creating personalized meal plans tailored to each user’s DNA, lifestyle, and health goals. Our app solves the issues many people struggle with when making meal plans:

  1. Accurately tracks user eating habits without customer input, improving convenience and helping users make healthier food choices
  2. Provides numerous options for meals, taking into account what’s inside the fridge and user data to help users save money. Users don’t have to go out of their way to purchase specific ingredients to start meal prep.
  3. Reduces food waste: uses the most of the ingredients in the fridge (tracks food quality through expiration dates) and reminds us of leftovers
  4. The app can be connected to dietitians and doctors for easy monitoring and communication

How we built it

We built NutriFlow AI by first collecting over 2000 images of food items, both on clean backgrounds and inside real fridges, to train a machine learning model for accurate food detection. Using Roboflow and YOLOv8, we trained and refined the model to recognize foods reliably in realistic settings.

Next, we integrated this detection data with LLaMA 3, an AI language model, to generate personalized meal suggestions based on each user’s age, allergies, dietary goals, and fridge contents. Finally, we designed a simple, user-friendly interface using V0 and React, connected to a Flask backend, allowing users to capture fridge snapshots and receive instant meal plans tailored to their health needs.

Challenges we ran into

We ran into two main challenges when developing our object detection software:

  1. Mass data collection: To create a dataset to train a model, we needed to collect thousands of images of various food items. Thus, we used a Gaussian Splatting technique to capture photos from every 1/16 frame in a video, a simple and quick solution to our data acquisition problem.
  2. Complex backgrounds: Our first model was based on clean background data, but for our applications, we needed a realistic simulation of a fridge. To solve this problem, we improved our dataset by collecting data on our objects inside our fridge. This allowed our model to identify objects even behind a complex background

Accomplishments that we're proud of + What we learned

In just two days, we were able to use machine learning software to create a model to identify 6 different food items (a ketchup bottle, an apple, a tomato, a cucumber, a celery, and a kiwi). Using over 2000 images taken from Gaussian Splatting techniques, this model could successfully identify all objects accurately, even in complex backdrops such as a fridge.

We also developed a functional and aesthetic front and back end for our user interface, which combines customer data and object detection to create useful and nutritious meal plans through Llama AI.

We learned developer skills like machine learning, Gaussian Spallating, system integration, and web development, and into one packaged project, as well as soft skills including teamwork, communication, project management, and especially time management.

What's next for NutriFlow AI

In the future, we plan to improve on the current systems…

  • Expand the dataset to support more food types and packaging variations
    • Improve detection in low-light and cluttered conditions
    • Add barcode scanning and expiry date tracking
    • Enable voice commands and AI chat for interacting with meal suggestions
    • Let users ask questions about the meals (e.g., alternatives, nutrition, preparation time) using a chatbot interface
    • Deploy the system to the cloud for multi-device access and smart home integration, and connect to dietitians & doctors

Built With

Share this project:

Updates