uPantry

Inspiration

The daily dinnertime dilemma is real: a stocked kitchen, but the mental block of "what to cook?" Searching for recipes, deciphering instructions, and the looming fear of food waste often lead to the convenience of takeout or dining out. This cycle results in wasted food, unnecessary expenses, and a missed opportunity for the joy of home cooking.

uPantry: Solving the Dilemma, Impacting the Planet

uPantry aims to revolutionize the home cooking experience. Our solution tackles the decision-fatigue that hinders many from making the most of their groceries. We envision:

• Effortless Meal Planning: An AI-powered recipe generator tailored to the user's existing pantry, minimizing the need for extra shopping.
• Intuitive Guidance: Step-by-step instructions that adapt to skill level, making cooking more enjoyable and successful.
• Waste Reduction, Savings Amplified: Our focus on using what's on hand directly combats food waste, leading to:
  • Significant cost savings for households.
  • Reduced environmental impact from food production and waste disposal.

The Power of Small Choices

We believe uPantry can empower individuals to make choices that benefit their wallets and the planet. By transforming the home cooking experience, we aim to:

• Demonstrate the cumulative environmental impact of reducing restaurant reliance.
• Highlight the financial benefits of mindful food utilization.
• Inspire a shift towards a more sustainable and enjoyable relationship with food.

What uPantry Does

uPantry eliminates the mealtime decision dilemma through a seamless AI-powered experience:

1. Snap and Analyze:

   • Users simply take a picture of the ingredients they have on hand.
   • uPantry's advanced image recognition technology analyzes the photo, identifying each ingredient with remarkable accuracy.

2. Recipe Inspiration:

   • Using Gemini's powerful AI, uPantry generates a curated selection of recipes tailored specifically to the identified ingredients.
   • Recipes prioritize minimizing additional purchases, helping maximize the use of existing groceries.
     

3. Step-by-Step Simplicity:

   • Once a recipe is chosen, Gemini generates clear, detailed instructions that guide the user through the cooking process.
   • Instructions adapt to the user's skill level, providing additional tips or simplifications where needed.

4. Global Impact Tracking:

   • uPantry calculates and displays the estimated financial savings, CO2 reduction, and tree planting equivalent for each home-cooked meal in real time.
   • A dynamic global counter demonstrates the collective impact of all uPantry users, fostering a sense of community and positive change.

The uPantry Advantage

• Effortless Inspiration: No more scrolling through endless recipes or feeling overwhelmed by choices.
• Waste Reduction: By turning existing ingredients into delicious meals, uPantry helps prevent food from going to waste.
• Skill Building: The intuitive, guided cooking process builds confidence in the kitchen.
• Savings: Reducing takeout and restaurant reliance saves money for users.
• Environmental Awareness: Real-time impact tracking encourages sustainable choices.

How We Built It

• Intuitive User Experience: uPantry leverages React.js to deliver a seamless interface. Users can easily capture images of their ingredients and are presented with visually appealing recipe suggestions.

• Robust Backend: The backend is built in Kotlin using the Spring Boot framework. This provides a well-structured and scalable foundation for handling image processing, recipe generation, and future feature additions.

• CI/CD Efficiency: We employ GitHub Actions to streamline development. This automates the process of testing, building, and deploying new updates or enhancements.

• Cloud Flexibility: uPantry is hosted on Google Cloud Platform's Cloud Run, ensuring the ability to scale effortlessly as our user base grows.

• Gemini AI at the Core The heart of uPantry's intelligence is Vertex AI, powered by the Gemini 1.5-pro-preview language model. We utilize Gemini for:

  • Accurate Ingredient Recognition: Gemini analyzes user-provided images, accurately identifying ingredients present.
  • Adaptable Recipe Generation: Gemini generates recipes tailored to the ingredients at hand, minimizing the need for extra grocery shopping. It also provides step-by-step instructions, adapting to the user's skill level.

Challenges We Faced

While the development process was overall rewarding, we did encounter a hurdle:

• Ingredient Highlighting: We aspired to integrate ingredient highlighting directly on the user's image. This would visually showcase the identified ingredients within the picture itself. While we explored potential solutions, time constraints prevented full implementation in this initial prototype.

Turning Challenges into Opportunities

This obstacle presented an interesting opportunity for future development. Highlighting ingredients directly on the image would enhance the user experience by providing a clear visual confirmation of what uPantry recognizes. We're excited to explore this feature in upcoming iterations.

Accomplishments that we're proud of

• Technical Proficiency: Rapidly mastered new technologies:

  • Seamlessly integrated Vertex AI's API, achieving impressive results out-of-the-box. Its ability to adapt suggestions based on constraints like "I don't have a stove" showcases advanced AI capabilities.
  • Intuitive use of Cloud Run's console UI and streamlined deployment with GitHub Actions for efficient CI/CD.

• Research-Driven Impact: Our research into the potential benefits of uPantry yielded compelling findings:

  • Financial Savings: Potential savings of $44.5 per meal for families who cook at home instead of ordering out underscores the significant financial impact.
  • Environmental Responsibility: Each home-cooked meal could reduce CO2 emissions by an average of 4.2 lbs, while the statistic about food waste in America highlights the need for action.

"Cook with uPantry. Save money. Save the Earth."

Beyond Code: Lessons in Teamwork and Impact

• Cross-Functional Power: Bringing together diverse roles – project management, user experience, backend, and infrastructure – was key to uPantry's successful launch. This project underscored the importance of a well-rounded team for rapid execution. Each team member's unique contributions made the whole greater than the sum of its parts.

• The Data-Driven Impact: We were inspired to transform an intuitive concern about food waste into quantifiable metrics. Researching financial savings and the CO2 reduction potential of home cooking gave our project tangible impact. Integrating this data into uPantry's user experience became central to our mission.

• From Concept to Cooking: It wasn't just theory – we put uPantry to the test! Collaborating in the kitchen, guided by our own app, became a practical lesson in how to design a truly user-friendly cooking companion.

• Embracing Flexibility: Initially, we planned to parse Vertex AI (Gemini) results into a rigid format to streamline recipe instruction linking. However, during testing, we discovered this limited the richness and adaptability of Gemini's responses.

• Creative Linking: To bridge the gap between initial meal suggestions and step-by-step recipe instructions within the first user interaction, we devised a solution using markdown links. We instructed Gemini to include these links at strategic points within its suggestions. This allowed the frontend to seamlessly extract the linked meal description and append it to the user prompt for the recipe generation process.

What's Next for uPantry

• AI-Powered Refinement: Leverage Retrieval-Augmented Generation (RAG) to integrate private recipe collections and a vast community recipe bank, enhancing accuracy and personalization.

• Personalization and Sharing:

  • Enable personalized cookbooks, recipe ratings, and modifications.
  • Implement easy sharing of recipes and shopping lists with friends and family.

• Beyond the Screen: Offer high-quality physical recipe book ordering, curated from personal or community selections.

• Community-Driven Growth:

  • Introduce a reward system for meals cooked, recipes contributed, and community engagement.
  • Develop a robust community recipe bank for exchange and inspiration.

• Emphasizing Impact:

  • Continue highlighting uPantry's economic and environmental benefits.
  • Partner with organizations to amplify the positive impact message.

Built With

• cloud-run
• kotlin
• kotlin/spring-boot-for-its-backend
• leverages-vertex-ai-(gemini)-for-image-analysis-and-recipe-generation
• react.js
• springboot
• vertexai