SMARTY!!!
Menu to add diet preferences
Fridge with ingredients
Empty fridge
Add ingredient manually
Add meal log manually
AI recipe details
Suggested AI recipies

AI Smart Fridge Assistant

Problem

Many people buy groceries without a clear plan, forget what ingredients they already have, and struggle to decide what to cook.
This leads to:

Food waste
Unhealthy last-minute meals
Time spent searching for recipes

Solution

Our AI Smart Fridge Assistant helps users decide what to cook using the ingredients they already have.

By analyzing a fridge image and generating recipes automatically, the system reduces decision fatigue and helps users make better use of their food.

Inspiration

As international students living away from home, managing food on a daily basis can be surprisingly difficult. It is common to buy groceries without a clear plan, forget what is already in the fridge, or struggle to decide what to cook with the ingredients available. This often leads to food waste, unhealthy eating habits, and unnecessary spending.

This project was inspired by a simple idea:

What if your fridge could actually help you decide what to cook?

Instead of manually checking ingredients or searching for recipes online, we wanted to create a system where a user could upload a photo of their fridge, allow AI to detect the ingredients, and instantly receive recipe suggestions based on what is already available.

Our goal was to build a tool that helps people cook smarter, eat healthier, and reduce food waste.

What it does

Our project is an AI-powered smart fridge assistant that helps users decide what to cook using the ingredients they already have.

Users can:

Upload a photo of their fridge
Automatically detect ingredients using AI
Generate recipe suggestions based on available ingredients
View nutrition insights and meal suggestions
Explore their ingredients through a 3D virtual fridge interface

Recipes are generated dynamically using AI, meaning the suggestions adapt based on the actual ingredients detected in the fridge.

How we built it

This project combines computer vision, generative AI, and interactive visualization.

1. Fridge Image Scanning

Users upload an image of their fridge. A vision model analyzes the image and detects visible food items.

2. Ingredient Extraction

Detected objects are converted into a structured list of ingredients that can be used by the recipe generation system.

3. AI Recipe Generation

We use Google Gemini to generate recipe suggestions based on the ingredients detected in the fridge.

4. Interactive Fridge Interface

Using Three.js, we built a 3D fridge interface where detected ingredients appear visually on shelves.

5. Nutrition-Based Recommendation System

Recipes are ranked based on how well they match the ingredients available in the fridge and their nutritional balance.

We approximate a recipe match score using the following formula:

$$ Score = \frac{\text{Ingredients Available}}{\text{Total Ingredients Required}} \times 100 $$

This prioritizes recipes that make the best use of ingredients the user already has.

Challenges we ran into

Ingredient Detection Accuracy

Fridge images can be complex. Items may be partially hidden, inside containers, or poorly lit, which makes detection more difficult.

Structuring AI Outputs

Generative AI typically produces free-form text. We needed to carefully design prompts and parsing logic to convert the responses into structured recipe data.

Building the 3D Fridge

Creating a responsive 3D fridge using Three.js required balancing visual detail with performance so that the interface remained smooth and interactive.

System Integration

We had to connect several components together: image input, ingredient detection, recipe generation, and the user interface.

What we learned

Working on this project gave us practical experience with several technologies and concepts:

Integrating generative AI into a real application
Prompt engineering for structured outputs
Building interactive 3D environments using Three.js
Designing AI systems that address everyday problems

More broadly, the project showed how AI can be used not just for automation, but also to support healthier habits and reduce waste.

Future improvements

Due to time constraints during the hackathon, several features remain part of our future roadmap.

Potential improvements include:

A gamified nutrition system with points and achievements
An AI health coaching chatbot to analyze user eating habits
Personalized nutrition analytics
Smart grocery recommendations based on missing ingredients
Machine learning models to learn user food preferences over time

These additions would evolve the project from a recipe assistant into a broader AI-powered nutrition companion.

Conclusion

This project explores how AI can improve everyday kitchen decisions. By combining image recognition, generative AI, and an interactive interface, we created a system that helps users make better use of the ingredients they already have.

Tools like this have the potential to reduce food waste, encourage healthier eating habits, and simplify meal planning for students and busy individuals.