Inspiration
The idea emerged from a common frustration—standing in front of the fridge, staring at random ingredients with no idea what to make with class in an hour. After spending too much time searching for recipes that didn’t fit it was clear there had to be a better way. What if you could simply take a picture of what’s in your kitchen and instantly get meal suggestions? No more wasted time—just quick, easy healthy recipes based on what you already have that keep to your dietary restrictions and goals.
What it does
Snap a picture of your kitchen ingredients, and our Azure AI and ML models do the rest—identify what’s in the photo, consider your dietary restrictions and lifestyle choices, and suggest meals using the spices and pantry staples you already have. The goal is to provide healthy recipes to students so they can focus on what matters.
How we built it
We built the front end using React, JavaScript, and TailwindCSS. To develop and train the machine learning image recognition model, we used Python, TensorFlow, and Roboflow. For the recipe suggestion feature, we leveraged Azure OpenAI for natural language processing, Bing AI for image processing, and Azure App Services for hosting.
Challenges we ran into
- The biggest challenges were learning, developing, and training models, as well as assembling the app’s components.
- Working with Next.js was like putting out fires every few hours.
- While working on the image ML, identifying single items was straightforward, but processing images with multiple items proved punishing and extremely time-consuming.
- Researched Azure products and services to ensure we found the best solutions for our needs.
- During assembly, progress often felt like three steps forward and five steps back.
Accomplishments that we're proud of
- We came to VTHacks with an idea and with minor imperfections reached our goals.
- We’re proud of how quickly we learned to develop and train two models.
- We gained valuable experience with Microsoft AI services and successfully integrated several into our project.
- We figured out how to deploy ChatGPT-4 as a training model and utilized Azure OpenAI services built on top of it.
What we learned
- We learned how to train computer visions ML.
- We learned how to train a Large Language Model.
- One of us learned React and TailWindCSS.
- How to use powerful Azure services.
- How to host React Apps on GitPages.
- How to deploy on Azure cloud.
What's next for Health PantryCam
If the users know they have certain health conditions we want to be able to predict their diet. We would also like to add a dynamic element to our ingredients page. As the user adds an ingredient, we want our app to display soft pop-up recipe suggestions. Lastly, we would like to train our image recognition model with more data and a larger selection of food products.
Log in or sign up for Devpost to join the conversation.