SnapChef

Our project uses AI-powered computer vision to make cooking easier and more sustainable for college students. Built using EyePop’s vision pipelines and AI agent pipelines.

Comment

🍳 SnapChef

SnapChef is an AI-powered smart kitchen assistant that turns photos of your pantry, fridge, or receipts into personalized recipe ideas, ingredient insights, and guided cooking experiences. By combining computer vision, OCR, and large language models, SnapChef helps reduce food waste, save time, and make cooking effortless.

💡 Inspiration

SnapChef is mainly geared toward college students and young adults who want to cook but simply don’t have the time or mental bandwidth to plan meals. Between classes, assignments, part-time jobs, and social commitments, many students default to takeout—even when they already have food sitting unused in their fridge.

We also saw how this lifestyle leads directly to food waste. Ingredients are forgotten, expire unnoticed, or get replaced by duplicate grocery runs—especially in shared apartments and dorm kitchens. This isn’t just expensive for students, it’s environmentally wasteful.

Students already rely heavily on YouTube cooking videos for quick meals, but the friction comes from figuring out what they can actually make with the ingredients they have right now. Decision fatigue, forgotten groceries, and expired food make cooking feel like extra work.

SnapChef removes that friction by turning a quick photo into instant, personalized cooking guidance—helping students save time, save money, and reduce food waste, all while fitting seamlessly into their fast-paced lifestyles.

🛠️ How We Built It

SnapChef is built using a modular, agent-friendly architecture:

  • Computer Vision (OpenCV + EyePop) for live camera capture, image quality checks, and ingredient detection
  • Blur detection (Laplacian variance) to block low-quality inputs before inference
  • EyePop Object Detection to identify pantry items, products, and shelf goods
  • OCR fallback (EyePop Text Detection) for receipts or low-confidence images
  • LLMs (Ollama / Gemini) for recipe generation, nutrition estimates, and reasoning
  • YouTube integration for sourcing and verifying cooking tutorials
  • Google OAuth for secure authentication
  • Email notification service for ingredient expiry alerts

The pipeline is designed to be cost-aware, privacy-first, and easily extensible for future agents and services.

📸 What It Does

SnapChef provides an end-to-end smart kitchen experience that learns and improves over time:

  • Scans your pantry, fridge, or receipts using a camera or image upload
  • Automatically detects ingredients and ignores irrelevant objects or people
  • Falls back to OCR when object detection confidence is low
  • Deduplicates and cleans detected items into structured, agent-ready output
  • Suggests recipes based on available ingredients
  • Learns from user behavior by analyzing:
    • Previously cooked recipes
    • Saved favorites
    • Pantry scan history
  • Uses this history to personalize future recipe recommendations
  • Filters recipes by meal type, time, cuisine, and health preferences
  • Estimates calories, macros, and assigns a health score
  • Predicts ingredient shelf life and urgency levels
  • Recommends optimal storage locations
  • Sends expiry reminder emails
  • Finds and verifies relevant YouTube cooking tutorials
  • Generates accessible, step-by-step text cooking guides
  • Allows users to manage pantry items manually
  • Saves scan history, recipes, and favorites

⚔️ Challenges We Ran Into

  • Image quality issues (blur, lighting, cluttered backgrounds)
  • Balancing cost vs. accuracy when deciding when to trigger OCR
  • Filtering noisy vision outputs while keeping useful data
  • Ensuring privacy by avoiding long-term image storage
  • Verifying YouTube videos actually matched the intended recipe
  • Designing prompts that ignore humans, prices, and irrelevant text

🏆 Accomplishments We’re Proud Of

  • Built a fully automated multi-stage vision + OCR pipeline
  • Implemented blur rejection before any paid inference
  • Achieved reliable ingredient detection with zero user labeling
  • Created clean, deduplicated, structured outputs for agents
  • Designed a privacy-first system with automatic data cleanup
  • Integrated recipe intelligence, nutrition, and video verification
  • Delivered a production-ready, extensible architecture in hackathon time

📚 What We Learned

  • Pre-inference quality checks dramatically improve system reliability
  • Prompt engineering is critical for vision and OCR accuracy
  • Multi-stage pipelines save cost without sacrificing performance
  • Accessibility (text-based guides) should be a first-class feature
  • Modular design makes experimentation and iteration much faster

🚀 What’s Next for SnapChef

  • Roommate Shared Pantry Mode
  • Shared ingredient inventory across roommates
  • Visibility into who bought what and what’s running low
  • Reduced duplicate grocery purchases
  • Collaborative meal planning
  • Mobile app support (iOS / Android)
  • Barcode scanning for packaged foods
  • Personalized nutrition goals and dietary restrictions
  • Smart grocery list generation
  • Calendar-based meal planning
  • Integration with smart fridges and IoT devices
  • Fine-tuned food-specific vision models

SnapChef turns everyday kitchen chaos into clarity—one snap at a time 📷🍽️

Built With

Share this project:

Updates