Freshify

Inspiration

Every year, households around the world waste over 630 million tonnes of food, costing approximately $1 trillion and contributing 8-10% of global greenhouse gas emissions. This isn't just an environmental crisis, it's hitting families financially as well. A typical American family could save around $1,600 annually by eliminating food waste. The problem exists across cultures and continents, with per capita food waste ranging from 100kg/year in North America to 5kg/year in Sub-Saharan Africa. Despite the magnitude of this issue, most existing solutions fail households by being prohibitively expensive, inconvenient, or simply ineffective. We were inspired by the journies of the average consumer, people who maintain healthy lifestyles but struggle with tracking their food inventory, resulting in over-purchasing, poor inventory management, and ultimately, food waste. Current solutions require too much manual input or come with a hefty price tag (like Samsung's Family Hub Fridge at $2,600-$4,000). We realized there was an opportunity to create something affordable, user-friendly, and effective technology that could integrate seamlessly into daily routines while making a meaningful impact on both wallets and the planet.

What it does

Freshify is a smart, AI-powered mobile grocery management system designed to streamline inventory management for fresh foods and groceries. Our solution bridges the information gap between purchase and consumption by leveraging AI technology to track food inventory without the tedious manual input that leads to user abandonment. Simply snap a photo of your receipt and grocery haul, and you'll be automatically reminded when your purchases are about to expire. The app will also give you recepie recommendations based on the items you purchased, helping you manage your fridge inventory.

How we built it

Freshify's frontend is built on React Native and Expo Go. The backend uses a custom Python script that leverages OpenAI to parse through receipts and images of groceries, as well as receipts. Images taken on the mobile app are sent using a FastAPI server to the backend, which is analyzed and returned back to the frontend. The data is then stored in a Supabase database for security and push notifications. In order to ensure accurate recognition, we engineered a powerful prompt with specified JSON output formatting to ensure accurate communication from backend to frontend. This prompt was then tested extensively using many different images of groceries and receipts to ensure accuracy.

Challenges we ran into

Due to time and resource constraints, we had to resort to using a general purpose LLM for image classification and expiry date identification. Under ideal circumstances, we would train our own model from real life images to make sure we accurately identify an image of a bunch of groceries as well as the expiry dates of all items in the image. In addition, the estimation for expiry dates are not perfect as you can only tell so much about an item's freshness from the image. Without knowing the exact date of expiration, the model can only interpret so much.

Accomplishments that we're proud of

We successfully developed a computer vision system that can recognize common grocery items with high accuracy while requiring minimal computational resources to run on mobile devices. We're also proud of creating a business model that aligns financial incentives for all stakeholders: users save money, grocery partners gain valuable data and additional sales, and Freshify generates sustainable revenue, all while reducing environmental impact. We learned to balance technical sophistication with practical constraints, focusing on high-impact features that could be implemented with relatively low cost but significant user benefit.

What we learned

Cursor AI is extremely powerful for implementing MVPs. With AI technology on the rise, "vibe coding" is common where most of the code is being generated by LLMs. Keeping this in mind, we still made sure to manually implement safe and cyberly-secure methods to ensure user data is kept private.

What's next for Freshify

In the near future, we want to implement our own computer vision model instead of relying on GPT. We also want to refine our recipe recommendation system to better account for user preferences, dietary restrictions, and skill level while still prioritizing items approaching expiration. We also want to develop functionality for multiple household members to contribute to and access a shared inventory, ensuring everyone is aware of available foods and expiration dates. As outlined in our long-term roadmap, we want to aim to develop a hardware system similar to a grocery store cashier aisle that accurately tracks food quantities in real-time, minimizing the need for manual scanning while increasing accuracy.

Built With

• expogo
• fastapi
• github
• openai
• python
• react-native
• supabase
• typescript