Pic2Plate — AI-Powered Metabolic Health Companion

Snap a photo of your meal, get instant diabetic-safe nutrition analysis. AI-powered GI scoring, sugar spike risk, and personalized meal coaching—built with Gemini 3. Scan. Understand. Eat smarter.

Comment

Inspiration

537 million adults worldwide live with diabetes. For them, every meal isn't just about taste—it's a calculation. Carbohydrates, glycemic index, portion sizes, sugar spike risks. Traditional nutrition apps require tedious manual logging, and most don't understand the nuances of metabolic health.

I wanted to build something that gives diabetics instant, intelligent answers just by taking a photo of their food.

What it does

Pic2Plate is an AI-powered nutrition assistant that transforms how people with diabetes approach meals:

📸 Instant Food Analysis

  • Snap a photo of any meal
  • Gemini 3 Flash Vision identifies every food item with confidence scoring
  • Get comprehensive nutritional breakdown in seconds

🧪 Metabolic Health Metrics

  • Net carbs calculation (total carbs minus fiber)
  • Glycemic Index assessment (Low/Medium/High)
  • Sugar spike risk prediction
  • Dietary warnings for common allergens and restrictions

🤖 Agentic Nutrition Coach

  • Interactive chat-based consultation with an AI dietitian
  • Ask questions like "Is this keto-friendly?" or "Create a 3-day meal plan"
  • Input your current glucose level for personalized recommendations
  • Watch the AI reason through your questions in real-time

👨‍🍳 Diabetic-Friendly Recipe Generation

  • Generate healthier recipe variants with lower GI alternatives
  • Automatic substitutions: almond flour, stevia, zucchini noodles
  • Integrated Google Search for curated web recipes

🖼️ AI Food Visualization

  • Type any dish name to get instant nutritional analysis
  • Gemini 3 Pro Image generates professional food photography
  • Perfect for meal planning before cooking

How I built it

Tech Stack

  • Frontend: React + TypeScript with Vite
  • AI Backend: Google GenAI SDK (@google/genai)
  • State Management: React hooks with localStorage persistence
  • Styling: Tailwind CSS with custom design system

Gemini 3 Integration

1. Gemini 3 Flash Preview (gemini-3-flash-preview)

  • Primary model for food image analysis
  • Multi-turn agentic chat sessions
  • Structured JSON responses with schema enforcement
const response = await ai.models.generateContent({
  model: 'gemini-3-flash-preview',
  contents: {
    parts: [
      { text: VISION_SYSTEM_PROMPT },
      { inlineData: { mimeType: 'image/jpeg', data: base64Image } }
    ]
  },
  config: {
    responseMimeType: 'application/json',
    responseSchema: foodScanSchema
  }
});

2. Gemini 3 Pro Preview (gemini-3-pro-preview)

  • Complex recipe generation with diabetic modifications
  • Google Search integration for web recipe grounding

3. Gemini 3 Pro Image Preview (gemini-3-pro-image-preview)

  • AI-generated food photography for text-based searches
  • Professional studio lighting aesthetic

Key Technical Features

  • Structured Output Schemas: Every AI response follows strict TypeScript-matching JSON schemas
  • Streaming Responses: Real-time chat with sendMessageStream for UX fluidity
  • Robust JSON Parsing: Multi-pass parser handles malformed responses gracefully
  • Error Categorization: Network, rate-limit, safety, and API errors handled distinctly
  • Session Context: Chat maintains full nutritional context across the conversation

Challenges I ran into

  1. JSON Reliability: Early versions occasionally received malformed JSON. Solved with a two-pass parser that extracts valid JSON even from markdown-wrapped responses.

  2. Maintaining Health Focus: Needed to carefully craft system prompts to prioritize diabetic-relevant metrics (glycemic index, net carbs) over general nutrition.

  3. Balancing Response Speed vs. Depth: Used Gemini 3 Flash for quick analysis and Gemini 3 Pro for complex reasoning like recipe generation.

  4. Agentic Session State: Keeping chat context while allowing users to update metabolic information (like current glucose) required careful prompt engineering.

Accomplishments that I'm proud of

  • Full Gemini 3 Family Integration: Leveraged Flash, Pro, and Pro Image models in a single coherent application
  • Production-Ready Error Handling: Comprehensive error boundaries, retry mechanisms, and user-friendly error messages
  • Real-Time AI Reasoning Visualization: Users can see the AI "thinking through" their questions
  • Diabetic-First Design: Every feature prioritizes metabolic health metrics over generic nutrition

What I learned

  • Schema-Enforced Responses are game-changing for building reliable AI applications
  • Agentic Patterns (multi-turn chat with context) create far more valuable experiences than single-shot queries
  • Model Selection Matters: Flash for speed-critical paths, Pro for complex reasoning
  • Prompt Engineering for Health: Medical/nutrition contexts require very specific guidance to avoid generic responses

What's next for Pic2Plate

  • Glucose Meter Integration: Connect with CGM devices for automatic readings
  • Meal History Analytics: Weekly/monthly carb trends and GI patterns
  • Personalized Goal Tracking: Set A1C targets and track progress
  • Multi-Language Support: Expand to serve diabetics globally
  • Healthcare Provider Sharing: Export reports for doctor consultations

Built With

Share this project:

Updates