Oasis

Image drop in selection
Manual text box selection
Checklist selection
AI generated ingredients found
Generated recipes
Opening animation

What inspired us?

As college students, the Oasis team saw in our peers and personally that food "micro wastes" often occurred due to a lack of knowledge on what to do with small, miscellaneous leftover scraps of food. These small micro wastes can accumulate quick. Using Oasis, users now have the perfect tool for discovering new ways to repurpose scraps into a delicious new meal. By simply inputting ingredients or taking a picture of your fridge, Oasis will deliver recipes catered to your preferences straight to your laptop or phone, allowing you to immediately get to cooking.

How we built it

With a React frontend and Express backend, these frameworks work together to analyze and compute data to bring to the user. When a user uses our web app, they can easily snap a photo of their fridge or leftovers which the express backend sends to Gemini. Gemini then analyzes the photo for ingredients. These ingredients are shown to the user for confirmation and can be changed if the model hallucinates. Finally, the user can generate recipes based on the food they already have by utilizing Gemini's JSON formatting structure to then display back onto the page.

Challenges we ran into

The model wouldn't comply to our restrictions and JSON formatting requirements, so we had to engineer a more rigorous prompt which took longer than we thought to get working. Hosting was also a problem; we wanted this web app to be mobile friendly and that means changing the CSS and hosting it. We ran into issues with Express and Vercel (even though they usually work well together) and the Gemini API but last second, we found a workaround.

What are we proud of?

During test runs of Oasis, our team actually ran into a bug in which our ingredient identification software was picking up fish sauce in a photo, and we could not understand why it was. Little to our knowledge this was not a bug at all. Oasis was able to identify fish sauce in the very top corner of the photo that the team had completely missed.

What we learned

This hackathon we had a great time with the Wild West theme, and we learned a lot more frontend development. We also never used Gemini APIs before, so we feel confident using it for our own personal projects.

How will we continue to improve Oasis?

The Oasis team hopes to continue reducing food waste not only by expanding our ingredients and recipes but possibly looking into integrating our software into smart refrigerators with cameras, so dinner ideas can be ready for you the moment you get home.