pantry-pro

Pantry-Pro tracks your kitchen by scanning receipts. It recommends smart recipes based on what you own and guides you step-by-step through cooking. Save money and reduce food waste!

Comment

Inspiration

The idea for Pantry-Pro came straight out of my daily life as a university student. Like most students, I was living on a budget, but I kept falling into the same frustrating cycle: I would buy a specific set of ingredients for a single meal, use only a fraction of them, and have no idea what to do with the leftovers. They would inevitably sit in the fridge and go to waste.

What it does

At its core, Pantry-Pro is a smart kitchen assistant designed to take the guesswork out of mealtime. Here is exactly how it works:

Smart Receipt Scanning: Instead of manually typing out every single item you bought, you simply snap a photo of your grocery receipt. The app reads the text and automatically stocks your digital pantry, keeping a real-time inventory of your kitchen.

Waste-Free Recipe Recommendations: When you are hungry but don't know what to make, Pantry-Pro looks at what you already have on hand—especially those tricky leftover ingredients from previous meals. It then suggests customized recipes you can cook right now, without needing another trip to the store.

Guided Cooking: Once you pick a dish, the app transforms into your personal sous-chef. It walks you through the entire cooking process with clear, step-by-step instructions, making it easy for anyone to learn how to cook from scratch.

How we built it

Smart Scanning via Gemini: I integrated the Gemini API to act as the brain behind the receipt scanner. It takes the raw, messy text from a grocery receipt photo and accurately translates those weird abbreviations into a structured list of ingredients.

Feature Development with Claude: For the more resource-intensive features and complex application logic, I used Claude to help write and structure the code. This collaboration dramatically sped up how quickly I could implement the app's trickier functionalities.

Data Management via AWS: The backbone of the project is an AWS database, which securely stores user profiles, real-time kitchen inventories, and recipe catalogs, communicating safely with the frontend using secure API keys.

Challenges I ran into

Receipt Parsing & Price Extraction: Grocery receipts are notoriously messy, with misaligned columns and weird abbreviations. Getting the Gemini API to consistently identify and map the correct price to the correct food item—without mixing up rows or hallucinating numbers—required significant prompt tuning and testing.

Sourcing Cohesive UI Animations: To give the app a premium feel, I used Lottie animations. However, finding a complete, open-source set that all shared the exact same visual aesthetic was incredibly difficult. Creating the missing custom Lottie files from scratch under a hackathon time crunch was a major design bottleneck.

Web to Native Bridging: Translating the Next.js web environment into a native Android app using Capacitor required extensive networking troubleshooting to bridge localhost ports to physical LAN IPs for testing.

Accomplishments that we're proud of

The Smart Receipt Scanner: Successfully taking messy, unpredictable real-world receipts and turning them into a clean, structured digital pantry using the Gemini API was a massive win for the app's core user experience.

Real-Time Family Sharing: We implemented a collaborative "Shared Household" feature that allows multiple users to sync to a single pantry. This keeps the whole family up-to-date on exactly what ingredients are available in real-time, completely eliminating the frustration of double-buying groceries.

Robust Recipe Database: We are incredibly proud of successfully provisioning our AWS DynamoDB backend and populating it with a high-quality, curated catalog of fully enriched recipes (complete with images, instructions, and dietary tags) during a short hackathon.

Dietary-Aware Recommendation Engine: We built a custom matching algorithm from scratch that doesn't just guess what to cook—it strictly respects the user's specific food requirements (like vegan or gluten-free) and mathematically ranks recipes based on what is actually in their kitchen.

The AI Chef Implementation: Integrating the AI Chef as a personalized kitchen assistant. It serves as a brilliant fallback, allowing users to ask for ingredient substitutions or custom cooking advice right when they need it most.

What we learned

Building Custom Claude Skills: We learned how to extend AI capabilities by creating and integrating custom tools and skills for Claude. This allowed us to build powerful, agentic workflows that helped us rapidly generate complex backend logic, database scripts, and our matching algorithm directly within our development environment.

Cloud Database Management: We gained hands-on experience designing and managing a NoSQL database using AWS DynamoDB. We learned how to provision tables, write scripts to bulk-seed our recipe data, and structure our queries to efficiently power our real-time matching engine.

API Security & Key Management: We learned the critical importance of keeping our application secure. By properly utilizing Next.js server-side API routes and environment variables, we ensured that our sensitive AWS credentials and Gemini API keys were never exposed to the client-side mobile application.

Integrating a Complex Mobile Stack: We learned how to seamlessly stitch together a diverse set of tools into a single, cohesive mobile app. Successfully combining a Next.js web framework, Capacitor for native Android deployment, AWS databases, and AI APIs taught us a massive amount about full-stack architecture

What's next for pantry-pro

Expanded Recipe Database: While we are proud of our current curated catalog, we plan to significantly scale up our AWS database. We want to add thousands of new recipes, covering an even wider array of global cuisines and highly specific dietary niches.

Enhanced UX with Lottie Animations: We want to continue upgrading the visual polish of the app by integrating a wider array of custom Lottie animations. Our goal is to ensure every micro-interaction, empty state, and screen transition feels fluid, native, and premium.

Deepened Interactivity: Cooking should be fun and engaging, so we aim to make the step-by-step cooking mode even more interactive. This includes adding built-in smart timers, swipeable gesture controls, and gamified cooking milestones to keep users coming back.

Commitment to Accessibility: We plan to implement robust accessibility features to ensure anyone can use the app comfortably. This includes adding voice-controlled navigation (perfect for messy hands while cooking!), improved screen reader support, high-contrast UI modes, and dynamic text sizing.

Built With

  • aws-databases:-aws-dynamodb-apis:-google-gemini-(vision-for-food/receipt-scanning-+-recipe-generation
  • aws-sdk-(dynamodb)
  • base-ui
  • capacitor-plugins-(camera
  • class-variance-authority
  • claude
  • clsx
  • css-frameworks-&-libraries:-next.js
  • git
  • gradle-(android-build)
  • html
  • incl.-gemini-2.5-flash-&-gemini-2.5-flash-image)
  • javascript
  • languages:-typescript
  • local-notifications
  • lottie-(lottie-web)
  • lucide-react
  • no-live-api)-tooling:-node.js
  • npm
  • openfoodfacts-(barcode-lookup)
  • postcss
  • progressive-web-app-(pwa)-cloud-services:-vercel-(hosting-+-serverless-api-routes-+-analytics)
  • react
  • shadcn/ui
  • speech-recognition)
  • tailwind-css
  • tailwind-merge
  • tesco-click-and-collect-(deep-link-out
  • testing-library-platforms:-capacitor-(android-native-shell)
  • tsx
  • v0
  • vitest
  • web-getusermedia-(camera)
Share this project:

Updates