ShelfLife: Save food, Save money!

ShelfLife is designed to help people track food expiration, reducing waste and promoting sustainable habits. Our app helps users manage their groceries efficiently by tracking expiration dates and sending timely notifications for items nearing expiry. Leveraging generative AI, the app also recommends recipes based on available ingredients and the user’s preferences, guiding them step-by-step through meal preparation.

This not only minimizes food waste but also enhances culinary convenience. ShelfLife aims to help users reduce food waste, save money, and make meal planning more convenient.

Inspiration

Each year, households waste billions of dollars due to forgotten groceries expiring. In the U.S. alone, 30-40% of the food supply is wasted, costing around $161 billion annually. This waste not only increases household expenses but also impacts the environment by contributing to greenhouse gas emissions and depleting valuable resources. We were inspired to develop ShelfLife to address these issues by offering an intuitive solution for tracking food expiration and reducing waste. Our goal is to promote sustainable habits while saving users money and minimizing food spoilage.

What it does

• Grocery Tracking: Add, view, and remove groceries with their expiration dates. Groceries are sorted by their expiration date to ensure users are notified about items that are expiring soon.
• Image Recognition: Use your device's camera to scan grocery items and identify them automatically using Google Gemini AI.
• Recipe Generation: Receive personalized recipe suggestions based on available ingredients and preferences using Google Gemini AI.
• Notifications: Get timely reminders when your groceries are nearing expiration to ensure you use them before they go to waste.
• Sustainable Lifestyle: Reduce food waste, save money, and promote eco-friendly habits with efficient grocery management and meal planning.

How we built it

We developed ShelfLife using a combination of technologies:

1. Frontend (React):

   • The user interacts with the app by adding groceries, taking pictures, or requesting recipe suggestions.
   • The app uses React Router for page navigation and fetch API to send HTTP requests to the backend.
   • react-webcam is used to capture images which are then sent to the backend for recognition.

2. Backend (FastAPI):

   • The backend receives requests for different operations:
     • Grocery Management: Stores, retrieves, and removes grocery items from the in-memory database via /groceries.
     • Image Recognition: Accepts image data, processes it with PIL, and sends it to Gemini AI via /identify.
     • Recipe Generation: Takes user inputs like ingredients and preferences, sends them to Gemini AI for recipe generation via /generate.
   • FastAPI uses the Generative AI SDK to communicate with Google Gemini AI, sending prompts and receiving AI-generated responses.

3. Google Gemini AI:

   • Image Recognition:
     • Receives the grocery image and identifies the item.
     • Sends back a recognized item from the accepted list of groceries (e.g., "apple").
   • Recipe Generation:
     • Receives selected ingredients and user preferences.
     • Returns AI-generated recipes with instructions based on the ingredients and preferences.

4. Frontend Rendering:

   • Once data is received from the backend (e.g., recognized grocery item or generated recipes), the frontend renders the content for the user.
   • React Markdown is used to render the recipes in a readable format.

Challenges we ran into

One of the main challenges was integrating Google Gemini AI for accurate grocery identification using images. Since AI-generated responses can sometimes be verbose or imprecise, we had to fine-tune the processing logic to only capture recognized grocery items from our predefined list. We also faced challenges with optimizing the user experience, particularly when rendering AI-generated recipe instructions in a readable format on the frontend. Ensuring smooth interaction between the frontend and backend, especially when sending and receiving image data, was another significant hurdle.

Additionally, we initially aimed to build a mobile app using React Native to extend ShelfLife's reach to mobile users. However, we encountered compatibility issues and performance bottlenecks when processing AI-generated content on mobile devices. While we made some progress, we decided to prioritize the web platform, where we had greater success, due to time constraints and the complexity of mobile development. Nonetheless, we hope to revisit the mobile app in the future as part of our ongoing development plans.

Accomplishments that we're proud of

• Successfully integrating FastAPI: This was my first time using FastAPI, and I’m proud of how smoothly I was able to implement it for this project. Not only did I manage to integrate Google Gemini AI for grocery identification and recipe generation, but I also realized that FastAPI is more efficient than the CORS setups I used before. It allowed me to streamline backend processes and handle complex data more effectively.

• Exploring React Native: While the final product is a web app, I’m proud of the effort I put into learning React Native. Although I faced challenges that led me to focus on the web version, the knowledge I gained in mobile development will be useful for future projects. I’m excited to eventually revisit React Native with this newfound experience.

• Sustainability-driven design: I’m particularly proud of how ShelfLife contributes to reducing food waste. The ability to track grocery expiration dates and provide timely notifications to help users make the most of their groceries directly ties into sustainability goals. This impact is something I’m proud to have built into the core of the app.

• Enhancing user experience: Another major accomplishment is optimizing the app’s user experience, especially when displaying AI-generated recipe instructions. Creating a seamless, intuitive interface while managing the complexity of integrating generative AI was a challenge, and I’m proud of the final outcome that makes meal planning simple and efficient for users.

What we learned

• Leveraging FastAPI for Scalability: This project introduced us to the capabilities of FastAPI, which proved to be a game-changer for handling backend operations efficiently. We learned how to scale API calls, integrate third-party services like Google Gemini AI, and process image data seamlessly. The speed and asynchronous nature of FastAPI taught us how powerful it can be, especially for data-heavy applications like ShelfLife.

• Enhancing AI Interactions for Real-World Use: While integrating Google Gemini AI, we discovered how critical it is to fine-tune AI responses for practical, real-world applications. Instead of just generating responses, we had to craft precise prompts, filter results, and ensure that only relevant data—like specific grocery names or recipes—were sent to the user. It highlighted how important prompt engineering is for optimizing generative AI.

• Adapting to Multi-Platform Development: We explored React Native as a way to extend ShelfLife into mobile platforms. While we eventually focused on the web application, the learning process gave us a strong foundation in understanding mobile development, component libraries, and the challenges of creating seamless experiences across different devices. The knowledge gained here will definitely help us when revisiting mobile development in the future.

• Designing for Sustainability and Convenience: Beyond just coding, we learned about the importance of designing for sustainability. Tracking food waste and providing recipe suggestions not only makes the app useful, but also helps users reduce their environmental impact. We understood how technology can directly contribute to eco-friendly lifestyles, and how crucial it is to design applications that blend convenience with positive environmental outcomes.

What's next for ShelfLife

Looking ahead, we plan to:

1. Expand AI capabilities: Further improve the image recognition model and expand the variety of grocery items that can be identified.
2. Add cloud storage: Implement cloud storage for grocery data, allowing users to sync their groceries across multiple devices.
3. Introduce voice commands: Add voice commands for adding groceries and requesting recipes for even greater convenience.
4. Collaborate with grocery stores: Explore partnerships with grocery stores to automatically import shopping lists and track expiration dates for purchased items.
5. Mobile app development: Develop a mobile version of ShelfLife to offer users on-the-go access to grocery management and meal planning tools.

Made by Sherwin, Avi and Mayank.

Built With

• fastapi
• google-gemini
• python
• react