-
Our frontend displays foods spinning around the center. Here’s the output after entering a pasta recipe with a Dairy-Free restriction.
Inspiration
After many late nights studying for our EECS classes and not having the energy to cook anything, we knew that there had to be a better way to eat better and healthier, avoiding takeout food every other day, without the hassle and headache of figuring out what to order and how to put it together in one complete recipe. That's when we came up with our idea of a tool that automates this entire process by turning any input, like a recipe or ingredient list along with any preferences, into a neat, smart and organized grocery list that is based on real-time data from local grocery stores.
What it does
The website allows users to input a recipe or a list of ingredients. It then parses the input to identify the ingredients, checks them against the user’s dietary preferences and budget constraints, and outputs a curated grocery list that fits their needs. This helps users plan meals efficiently and avoid unnecessary purchases.
How we built it
We built the frontend using React, with a clean interface featuring a text input and a button for processing recipes. The backend is designed to analyze the input using AI-powered parsing (through a basic ASI module) to detect ingredients and cross-reference them with user preferences and budget. Here, this is where the ingredients needed are extracted from a long, unorganized input from the user as well as important dietary restrictions and budgets. We also use the processing power of ASI:One to apply these dietary filters on the recipe/ingredient list, and then start the query for the Kroger API for matching products. The system then produces a structured actionable grocery list that is ready for shopping that is mailed straight to the user with the help of AgentMail. This way, the user can interact with the agentic AI to address any concerns or make changes to their grocery list if needed. There are also future use cases of this powerful mailing agent that aren't as apparent now but will be explained in the future section.
Challenges we ran into
We had to figure out how to accurately parse natural language input from users to identify ingredients, including dealing with ambiguous phrasing and variations in ingredient names. Integrating the backend AI parser with the frontend while keeping the process fast and responsive was also challenging. Additionally, we had some challenges integrating the backend process with the Kroger API as we sometimes ran into unexpected product matches or gaps in our data.
Accomplishments that we're proud of
We successfully created an functioning end-to-end pipeline that goes straight from user input to processed grocery suggestions, fully integrating AI parsing with the React frontend. The website is intuitive, interactive, and demonstrates clear utility for users who want to optimize their grocery shopping experience. We were able to implement budget-conscious shopping, which is hard to find in common recipe apps or websites online. Lastly, we were able to create a fast and responsive UI which clearly shows results to users through a concise and simple emailing system.
What we learned
We gained hands-on experience connecting a React frontend to a custom backend for natural language processing tasks. We also learned how to design a system that balances user experience with data processing efficiency, and how to handle edge cases in ingredient parsing. Additionally, we were able to gain experience on how to integrate third-party API's like Kroger to existing code and handle any potential issues that we had with it.
What's next for Automated Grocery Tracker
What's next for Automated Grocery Tracker Although we are still incredibly proud of what we were able to make in such a short time, our plans coming into this hackathon were much more ambitious than what we ended up developing. In the future, we want to implement the Snapchat Spectacles so that users can just look at their fridge or current ingredients and the object detection tools built within the Lens Studio platform will be able to determine the existing ingredients. We would then send this over to our backend system along with also giving users the capability to talk with the integrated AI on the Spectacles to give detailed feedback about their preferences and budget. Lastly, rather than just sending the grocery list to the users, we would aim to implement an automatic ordering system so that the user does not have to worry about parsing through the websites of the potentially multiple grocery store websites and buying items individually. This way, the entire frontend website can be avoided and result in an even more seamless and easy experience for users to get exactly the ingredients they want.
Log in or sign up for Devpost to join the conversation.