-
-
Foodi Logo
Inspiration
We wanted to create a project that implemented the categories we were all interested in, like UI/UX, Machine learning, AI, and Data Analysis.
One of our group members mentioned how we could do a project on generative AI, which sparked the discussion about what we could have it generate. Our first thought was to scan shoes and find their price via scanning, however, after further discussion, something even more elegant came up. One of our members was hungry and he and his friends really care about the protein count in their food and always had to ask or search up the macros of his food before getting something. So, he thought that it would be awesome if he could scan food and get the nutrient facts of it just by hovering his phone over it. The perfect merge of all of our groups' computer science interests was born, and now we have our proudest project to show for it!
What it does
Our web app, Foodi™️, can scan common types of food and recognize the food item, as well as give essential nutritional facts straight from a food label database. It’s easy to use and get running, and we have plans to continue working on it and hope for it to be useful to others hoping to get nutritional facts on the go from items that may not be in a package.
How we built it
Our group employed a combination of deep learning and web development technologies to develop a food classification system. On the machine learning side, I used TensorFlow and Keras, two powerful libraries for building and training deep learning models. Specifically, I leveraged MobileNetV2, a lightweight convolutional neural network architecture pre-trained on the ImageNet dataset, as the base for my model due to its efficiency and effectiveness in feature extraction from images. The model was trained on the Food-101 dataset, which consists of 101,000 images across 101 food categories, using data augmentation techniques such as rotation, zoom, and flips for improved generalization.
For the model's deployment, my group created a REST API using Flask, a lightweight web framework in Python, enabling easy integration and communication between the client-side and the server-side. Additionally, they integrated this Nutritionix API with a Node.js server, utilizing Express.js for handling HTTP requests and facilitating the interaction between the front end and the machine learning model. This setup allowed for seamless processing of image data, where images are sent to the server, classified by the model, and the predictions returned to the client.
Challenges we ran into
The model took a while to train using the MobileNetV2 neural network as the base model, approx 7-8hrs were needed to train the model and then we had to combine the trained model to the front and back end systems. This caused some dependency issues, with merging the front and back ends and getting the nutrition fact data. There were also some hiccups with integrating the trained AI model with the server and the front end to work with the phone formatting.
Accomplishments that we're proud of
We are proud to have gotten the front-end working integrated with the back-end interface and to have the project functioning as intended. Managing to convert and preprocess the base64 image for our AI model. Training the MobileNetV2 AI model from scratch effectively with accurate results and implementing an API for nutrition facts working alongside it.
What we learned
We learned from each of our specialties, UI/UX design, web development, Machine Learning (specifically about how convolutional neural networks work and AI libraries such as Tensorflow and Keras), and data science, and congregated them to create something we found special. We learned a ton from each other and the importance of each of our individual experiences.
What's next for Foodi
More training for our Foodi AI, fine-tuning the model and neural network and in the future we plan on implementing more images of various food items and our web app will continue to seamlessly learn and scan food items.
Log in or sign up for Devpost to join the conversation.