Fridge Genie

Inspiration

You know that feeling when you open your fridge at 7 PM and just... stare? Yeah, we've all been there. We kept throwing away wilted vegetables and expired ingredients because we'd forget what we had. Or we'd impulse-buy groceries we didn't need. The worst part? Scrolling through recipe apps while standing in front of a full fridge, feeling completely uninspired. So we thought—what if your fridge could just... tell you what to make? Like having a personal chef who knows exactly what's in your pantry. That's how FridgeGenie was born.

What it does

FridgeGenie is stupid simple to use (in a good way!). You snap a photo of your fridge. That's it. Our AI scans the image, detects every ingredient it can see—milk, eggs, that random bell pepper you forgot about—and lists them out with little confidence badges. Then you just ask: "What can I make for dinner?" And boom. The AI chat responds like a real chef would. It suggests recipes based on exactly what you have, shows you cooking times, dietary tags, the whole deal. No more guessing. No more wasted food. Just a conversation with an AI that actually gets it. Oh, and if you find something you love? Save it to your favorites. Done.

How we built it

Honestly? We dove in headfirst and learned as we went. Frontend: Next.js 14 with React 19 and Tailwind CSS. We wanted it fast, mobile-friendly, and camera-first. The UX is built around making everything feel effortless—big buttons, clean ingredient chips, smooth interactions. AI Brain: Google Gemini 2.5 Flash powers everything. The vision model detects ingredients from photos (with scary-good accuracy, btw). The chat model handles the conversational recipe recommendations. We're streaming responses with the Vercel AI SDK so it feels super responsive. Search Engine: Elasticsearch with hybrid search. We combine semantic search (embeddings) and keyword matching to find recipes that actually match your ingredients. Not just "close enough"—like, actually what you can cook. State Management: Zustand keeps track of your pantry ingredients across the app. Lightweight, fast, no headaches. Deployment: Vercel for the frontend. Elasticsearch is running on Elastic Cloud. Environment variables everywhere (shoutout to .env files doing the heavy lifting). We iterated a lot. Like, a lot. The first version was clunky. But each sprint got us closer to something that felt... right.

Challenges we ran into

Oh man, where do we start? 1. Ingredient Detection Accuracy Gemini Vision is incredible, but sometimes it gets... creative. It once detected "abstract art" in a photo of leftover pasta. We had to add confidence thresholds and let users manually edit the ingredient list before searching. 2. Elasticsearch Setup Setting up hybrid search was a rabbit hole. Balancing semantic embeddings with keyword matching? Tuning relevance scores? Yeah, that took way longer than expected. But when it clicked... chef's kiss. 3. Streaming AI Responses Getting Gemini to stream conversational responses through the Vercel AI SDK was tricky. We wanted it to feel like chatting with a real person, not waiting for a wall of text. The streaming API helped, but wrangling the state updates? Brutal. 4. Mobile Camera Integration Making the camera capture feel native on mobile browsers was harder than we thought. Different devices, different permissions, different quirks. We eventually nailed it, but not without some late-night debugging sessions. 5. Recipe Data Quality We needed a solid recipe dataset with proper metadata (cook time, dietary tags, ingredient lists). Finding one that was clean, structured, and searchable? Took forever. We ended up curating and normalizing data from multiple sources.

Accomplishments that we're proud of

Honestly? The whole thing feels like a win, but here are the highlights: The ingredient detection actually works. Like, really well. Watching someone snap a fridge photo and see it recognize "half an onion" and "sriracha bottle" never gets old. The chat feels natural. It's not robotic. It pauses, thinks, suggests, and remembers context. That took work. Hybrid search is fast. Sub-second results, even with complex ingredient queries. Elasticsearch + embeddings = magic. People actually want to use it. We showed it to friends and family, and they were like, "Wait, can I use this now?" That validation hit different. We shipped something we're genuinely excited about. Not just a demo. A real app that solves a real problem.

What we learned

So much. Like, so much. Technical: Gemini Vision is powerful, but you need guardrails (confidence scores, user confirmation). Hybrid search is better than pure keyword or pure semantic. Combining both is where the magic happens. Streaming AI responses dramatically improves perceived performance. Users don't mind waiting if they see progress. Next.js 14's App Router is fantastic once you get used to it (but the learning curve is real). Product: People care about speed more than perfection. A fast, 80% accurate result beats a slow, 100% accurate one. Mobile-first is non-negotiable for this use case. Nobody wants to transfer fridge photos to their laptop. Editing detected ingredients is crucial. Users need control, not just automation. Visual feedback (confidence badges, ingredient chips) builds trust. Team: Iterate fast, validate early. We killed features that sounded cool but didn't solve the core problem. Documentation matters. Future us (and our teammates) thanked past us for writing clear comments. Coffee helps. A lot.

What's next for Fridge Genie

We're just getting started. Here's what's on the roadmap: 1. Intelligent Ingredient Substitutions Missing one ingredient? The AI will suggest pantry swaps with explanations. Like, "No buttermilk? Use milk + lemon juice—here's why it works." 2. Cost Estimator Show how much it'd cost to buy missing ingredients. "Complete this recipe for $4.23 at your local grocery store." 3. Interactive Cooking Timeline Visual timeline showing prep time, active time, and what you can do in parallel. Think: "While the oven preheats, chop the onions." 4. Shopping List Generator One-tap export of missing ingredients to a shopping list (maybe even integrate with grocery delivery APIs). 5. Dietary Preference Learning The more you use it, the better it gets at suggesting recipes you'll actually love. AI that remembers you're vegan or hate cilantro? Yes please. 6. Community Recipe Uploads Let users contribute their own recipes. Build a crowdsourced database of real, home-cooked meals. 7. Multi-language Support Because food is universal. Spanish, French, Mandarin... we want everyone to cook smarter. The dream? Make FridgeGenie the go-to app for anyone who's ever stood in front of their fridge and thought, "What the heck do I make?" We're getting there. One fridge photo at a time.

Built With

• elastic-cloud
• elasticsearch
• eslint
• google-gemini-2.5-flash
• google/generative-ai
• lucide-react
• next.js-14
• node.js
• postcss
• react-19
• tailwind-css
• typescript
• vector-embeddings
• vercel
• vercel-ai-sdk
• zustand