nutrivision AI

nutrivision : Your AI dietary guardian. Uses Gemini to scan labels & menus, finding hidden allergens standard apps miss. It doesn't just read; it reasons to keep you safe.

Comment

This is a comprehensive project description tailored for a Hackathon Submission Page (like Devpost), a GitHub README, or a **Pitch Deck.

It uses a narrative structure that judges love: Problem $\rightarrow$ Solution $\rightarrow$ Tech Stack $\rightarrow$ Future.

🥗 nutrivision AI: The AI That Protects Your Health & Heals the Planet

Tagline: A multimodal AI assistant that safeguards users from allergens, reduces household food waste, and automates humanitarian food donation logistics.

💡 Inspiration

We live in a world of paradoxes. On one hand, 33 million Americans suffer from food allergies, navigating a minefield of confusing menus and hidden ingredients every time they eat out. On the other hand, we waste 40% of our food supply, while millions face food insecurity.

We asked ourselves: What if the same AI that protects a user from a peanut allergy could also protect the planet from waste?

That question led to nutrivision AI, a single, cohesive application powered by Google Gemini that serves as a *Guardian for your health, a Chef for your fridge, and an Angel for your community.

🚀 What It Does

nutrivision uses advanced Multimodal AI (Text + Vision) to operate in three distinct context-aware modes:

🛡 Mode A: Guardian (The Safety Engine)

For users with allergies or dietary restrictions (Vegan, Keto, Halal), dining out is stressful.

  • How it works:* Users simply snap a photo of a menu. nutrivision doesn't just read the text; it applies geo-cultural reasoning.
  • The "Wow" Factor:* If a user with a peanut allergy is at a Thai restaurant, the AI flags dishes like Pad Thai even if "peanuts" aren't explicitly listed on the menu, citing the high risk of cross-contamination in that cuisine.

👨‍🍳 Mode B: Chef (The Waste Reducer)

We often buy groceries, forget them, and throw them away.

  • How it works:* The user takes a photo of their open fridge or pantry.
  • The Intelligence:* The AI identifies ingredients and analyzes their freshness state (e.g., "Spinach - Wilting, High Urgency"). It then generates a recipe specifically designed to use up the ingredients that are about to expire, prioritizing waste reduction over preference.

😇 Mode C: Angel (The Social Logistics Engine)

When individuals or businesses have too much food, they want to donate but fear liability or logistics.

  • The Safety Audit:* The user photographs the surplus food. The AI strictly audits it for safety (rejecting open/eaten food, accepting sealed/whole items).
  • Smart Routing:* Using the Google Maps integration, the app automatically routes food to nearby **Soup Kitchens* , solving the "last mile" logistics problem instantly.

⚙ How We Built It

The architecture is a "Chain-of-Thought" pipeline that converts visual data into structured JSON actions.

  1. Frontend: Built with Next.js and Tailwind CSS for a responsive mobile-first experience.
  2. The Brain (Gemini 2.5 Pro): We feed the image and a "User Context String" (fetched from Firebase) into Gemini.
    • Prompt Engineering: We designed a dynamic system prompt that switches modes based on visual triggers (Menu vs. Fridge vs. Bulk Food).
    • Output: Gemini returns strict JSON, which our frontend parses to render specific UI components (Maps, Recipe Cards, Warning Shields).
  3. Database: Firebase Firestore stores the complex user profiles (allergies, dislikes) that act as the filter for the AI's decision-making.
  4. Logistics: We integrated the *Google Maps Embed integration, dynamically injecting search queries generated by Gemini (e.g., *"Homeless shelters in Bhubaneswar accepting cooked food") to render real-time donation points.

🧠 Challenges We Ran Into

  • Hallucinations vs. Safety:* Early on, the AI would sometimes guess ingredients. We solved this by implementing a "Strict Caution Protocol" in the System Instructions—forcing the AI to flag any ambiguity as a potential risk rather than guessing "Safe."
  • The "Context" Problem:* A photo of "Pasta" is safe for a vegetarian but unsafe if cooked in chicken broth. We added the Description Box feature, allowing the user to add verbal context that the AI weighs heavily in its final verdict.
  • JSON Consistency:* Getting the LLM to output clean JSON for the app to parse was tricky. We solved this by providing "One-Shot Examples" in the prompt, locking the model into a specific schema.

🏆 Accomplishments That We're Proud Of

  • The "Angel Mode" Routing:* Watching the map update dynamically felt like magic. It proved the AI understood the physics of the food.
  • Real-Time Latency:* We optimized the image compression to get analysis results back in least time possible, making it viable for standing in line at a restaurant.

🔮 What's Next for nutrivision AI

  • IoT Integration:* Connecting directly to "Smart Fridges" to scan inventory automatically without taking a photo.
  • Verified Charity Portal:* Building a dashboard for charities to update their "Real-Time Needs" (e.g., "We need milk today"), which nutrivision would prioritize in the matching algorithm.
  • AR Overlay:* Using Google ARCore to project the "Safe/Unsafe" highlights directly onto the physical menu through the camera lens, rather than on a static photo.

Built with 💚, ☕, and Google Gemini.

Share this project:

Updates