Inspiration
A lot of people don't know what to do with all the ingredients stored in the fridge or are clueless to how many different dishes their subset of ingredients can produce. ReciPic solves this issue by providing the user with a recipe that they can make with their ingredients or from buying a few more.
What it does
ReciPic is both a mobile app and Chrome extension. After the user logins using Firebase for authentication, the user is redirected to a camera from which they can snap a series of pictures of their fridge. This images are stored in their own personal folders on Firebase, from which our model processes, identifying the user's current ingredients and prompts Gemini for a recipe. The user is then displayed this recipe which may be complete or incomplete. If the recipe is incomplete, this is where the Chrome extension comes into play, which buy the remaining ingredients online from Walmart automatically.
How we built it
The frontend of the app was built in TypeScript and JavaScript using Expo and React Native. We then took a model from Ultralytics and trained it on a dataset of different ingredients on the RCAC servers. The Chrome extension was styled with CSS/HTML and programmed with JS. Lastly, the backend of this project was built in Python using Flask, and used a LLM to query recipes from a list of ingredients.
Challenges we ran into
We had trouble sending the images the user took from their phone to the model. We tried sending the images directly to the server hosted by Flask but kept running into network request issues, so we eventually decided to use Firebase as an intermediary and to store the images.
Accomplishments that we're proud of
Our idea was ambitious because we were trying to build both a mobile app and a Chrome extension, and we are proud of pulling both of those off. Additionally, we are proud of training our own model instead of wrapping a pre-trained one.
What we learned
Since we were inexperienced with these technologies, we learned a lot about these technologies. We learned a lot about app design and integrating a user interface with functionality. We also learned a lot about AI and how a model is trained.
What's next for ReciPic
Our next steps is to create a better model that can detect multiple ingredients from one image, instead of having to take many different images. Furthermore, we would like to display multiple recipes.
Log in or sign up for Devpost to join the conversation.