PantryPilot

Make Every Ingredient Count

Comment

Inspiration

I often open my fridge to see half eaten pieces of food, wondering why this is such a habit, leaving things uneaten, quite tempting to dispose of.

What it does

  • Budget Tracking
    You enter your monthly food budget once and PantryPilot continuously compares what you’ve spent to the estimated value of everything on your shelves.
  • Barcode-Powered Inventory
    Point your phone’s camera at any UPC barcode, and we automatically pull in product names, nutrition facts, and estimate expiry dates via the free Open Food Facts API.
  • Waste-Reduction Tips
    Get contextual advice (“turn broccoli stems into pesto,” “regrow green onions from scraps”) to squeeze every last drop of value from your ingredients
  • Local Price Optimization
    (Stubbed for future work) We’ll recommend cheaper substitutes from nearby stores or flyer deals so you never overpay for that last missing item.
  • “What Can I Eat?” Recommender
    Instant list of full meals you can make right now

How we built it

  1. FastAPI Backend
    • Loaded secrets from .env (OpenAI key, OFF base URL).
    • /inventory endpoint fetches product data, nests nutrition under nutriments, and auto-estimates a six-month expiry.
    • AI endpoints (/mealplan, /whatcanieat, /tips) craft precise prompts and call GPT-4o-Mini via the new OpenAI v1 client.
    • In‐memory data stores for rapid prototyping (swap in a real DB later).
  2. React + JSX Frontend
    • A reusable <CodeScanner> component uses a ref to mount Quagga.js for live barcode detection.
    • Axios-driven services point to REACT_APP_API_URL, keeping fetch logic isolated.
    • Four modular components: Inventory, BudgetTracker, MealPlan, and Tips for clear separation of concerns.
  3. Wireframes & Iteration
    • Rapid prototyping in Figma, then pixel-perfect builds in VS Code.
    • End-to-end testing with family-style pantries to tune prompts and UI flows.

Challenges we ran into

  • Quagga Initialization Errors
    Crashed on setAttribute until we switched to a ref-based mount and gated init until the DOM node existed.
  • Hook Pitfalls
    An async callback in useEffect dumped warnings—moving async logic inside the effect callback cured it.
  • Flaky API Data
    Open Food Facts occasionally omits fields, so we built graceful fallbacks (“Unknown product,” six-month expiry) to avoid UX jank.
  • Prompt Tuning
    Balancing nutrition, cost, and waste in our meal-planning requests took dozens of micro-edits and real-world testing.

Accomplishments that we’re proud of

  • Live barcode scanning in the browser with reliable back-camera support.
  • Generating multi-day, budget-aware meal plans from a simple pantry list.
  • Seamless budget vs. inventory value tracking in real time.
  • Building a full-stack prototype in under 48 hours—front to back.
  • Pivoting mid-hackathon from a paid UPC service to the free Open Food Facts API without losing functionality.

What we learned

  • The art of prompt engineering: precision matters when you ask an AI for balanced, cost-sensitive menus.
  • Quagga.js quirks around live streams and camera permissions in React.
  • How to glue FastAPI, OpenAI v1 client, and React into a cohesive, maintainable codebase.
  • The importance of graceful degradation and sensible defaults when external APIs fail.
  • Modular component architecture makes rapid iteration feasible under tight deadlines.

What’s next for PantryPilot

  • Real-time Price Feeds: Integrate Instacart/Walmart APIs (or flyer OCR) for live substitute recommendations.
  • Smart Notifications: Push alerts via email/SMS when high-value items are nearing expiry.
  • Personalized Profiles: Let the AI learn your dietary preferences, allergies, and taste history.
  • Pilot Program: Roll out to 10 local households for real-world feedback and impact measurement.
  • UX Polishing: Add dark mode, accessibility enhancements, and multi-language support.
Share this project:

Updates