FridgeMaster

Inspiration

Every family knows the frustration: buying fresh groceries only to discover them growing mold weeks later in the back of the fridge. We wanted to solve this universal problem using AI, especially after discovering Perplexity Sonar API's powerful vision capabilities. Beyond just tracking, we envisioned a system that would actively alert users when food is about to expire, preventing waste before it happens. Current calorie-tracking apps require manual photo-taking of each meal - our solution automatically tracks everything in your fridge and calculates nutrition when suggesting recipes.

How We Built It

Over a month, we developed SmartFridge using Flask, MongoDB Atlas, and React, with Perplexity Sonar API at its core. We implemented MongoDB's multi-modal image vector search to store and compare food images efficiently, creating a RAG (Retrieval-Augmented Generation) system that learns from previous identifications. Every item placed in the fridge is registered and stored for future reference, building a comprehensive food database unique to each household.

Midway through development, we discovered Samsung's similar smart fridge project - but realized most families won't buy a $3000 fridge just for this functionality. Our solution: a camera sensor system that works with any existing fridge, capturing snapshots when hands approach or leave. The system includes expiration alerts that notify users when items are approaching spoilage.

Challenges We Overcame

The biggest hurdles were mastering prompt engineering for Perplexity to return consistent, structured JSON responses for both item identification and expiration date estimation. We spent considerable time designing efficient data structures that could handle image vectors, metadata, and usage patterns while maintaining fast query performance.

Reducing latency was critical - we implemented MongoDB's vector indexing to create an intelligent memory system. The hardware integration presented unique challenges: distinguishing between "putting in" versus "taking out" actions required careful analysis of hand positions and movement patterns.

Future Vision

We're planning to track users' food consumption patterns and grocery shopping frequency to automatically generate personalized shopping lists, making the entire food lifecycle smarter and more efficient.

Built With

• flask
• javascript
• mongodb
• mongodb-atlas
• perplexity
• pydantic
• python
• radix-ui
• react
• react-18
• tailwind-css
• typescript
• vite