CulinarIA

Your personal AI-powered recipe manager and cooking companion. Organize, discover, and master recipes with smart features and premium cooking tools.

Comment

CookShelf: AI-Powered Cooking Companion

Problem Statement

Cooking content creators and enthusiasts face daily friction: they discover recipes through YouTube, Instagram, and blogs but have no structured way to save, organize, and cook them. When cooking time arrives, they're juggling a phone with flour-covered hands, scrolling back through videos to find specific steps.

Existing tools don't solve this:

  • Pinterest/bookmarks: Save links but offer no cooking experience
  • Generic recipe apps: Can't import from video descriptions or social posts
  • Notes apps: Unstructured, not searchable, no hands-free mode

CookShelf bridges the gap between discovering a recipe and actually cooking it.

Solution Overview

CookShelf is your AI-powered cooking companion built with Google Gemini's multimodal ecosystem and React Native.

Audience Problem CookShelf Solution
Can't save recipes from creator URLs Paste any URL — Gemini extracts the recipe instantly
Physical cookbooks can't be searched Scan any page with camera — Gemini Vision structures it
Hands are busy while cooking Voice assistant powered by Gemini 2.5 Flash Native Audio
No visual reference for unfamiliar dishes AI generates step-by-step images with Gemini Imagen
Forget ingredients at the store Smart pantry + auto-generated shopping lists

Core Features (5 Tabs in Bottom Navigation)

1️⃣ Extract Tab (Scanner)

  • Scan Mode: Take photo or upload from gallery - Gemini Vision extracts recipe from cookbooks, magazines, screenshots
  • Link Mode: Paste URL from any recipe blog, YouTube video description, TikTok, Instagram
  • Search Mode: Search billions of recipes by dish name, with local cookbook integration
  • Trending searches: Quick access to popular recipes (Avocado Toast, Chicken Tikka Masala, etc.)

2️⃣ Recipes Tab (Recipe Library)

  • My Cookbook: Searchable recipe library (51+ recipes in database)
  • Match sorting: Sort recipes by pantry ingredient match percentage
  • Ingredient matching: Shows how many ingredients you already have
  • Recipe details: Full ingredients, step-by-step instructions, nutrition info, AI-generated step images

3️⃣ Kitchen Tab (Kitchen Manager)

  • Pantry Mode: Track ingredients at home organized by aisle (Meat & Seafood, Dairy & Eggs, Produce, etc.)
  • Shop Mode: Smart grocery list grouped by category
  • Quick Add Essentials: One-tap staples (eggs, milk, butter, flour, salt, sugar, oil, etc.)
  • AI Ingredient Parsing: Natural language input ("2 cups flour, 3 eggs") parsed into structured data
  • Smart Merging: Combines duplicate ingredients from multiple recipes automatically
  • Move to Pantry: After shopping, check off items and transfer to inventory

4️⃣ Chef Tab (Chef Discovery)

  • AI Recipe Suggestions: Get recipe ideas based on your current pantry inventory
  • Cuisine Filters: Italian, Mexican, Asian, American, Indian, Mediterranean, etc.
  • Meal Type Filters: Breakfast, Lunch, Dinner, Snacks, Desserts
  • Pantry Badge: Shows your current inventory count
  • Smart Matching: AI suggests recipes you can make with ingredients you already have

5️⃣ Profile Tab

  • Subscription Management: Free, Starter, Pro, Ultimate tiers
  • Usage Tracking: Searches, scans, voice minutes, AI images, suggestions, nutrition analyses
  • Billing History: View past purchases
  • Settings: App preferences
  • Feedback: Submit feedback to developers

Monetization (RevenueCat)

Subscription Tiers (configured in RevenueCat):

Tier Key Features
Free Limited feature access
Starter 50 Recipe Storage, Unlimited Scans, Smart Pantry
Pro Unlimited Recipes, Nutritional Analysis, Web Search Integration
Ultimate All Pro Features, 480 Min Smart Chef (Voice), Priority Support
  • RevenueCat SDK integrated via react-native-purchases (v9.7.6)
  • Full webhook lifecycle handling (200+ events tracked in database)
  • Usage tracking per billing period
  • Monthly subscription model with trial support

Technical Architecture

[React Native App (Expo 54)]
       |
       |-- REST API --> [Node.js/Express Backend (Google Cloud Run)]
       |                    |
       |                    |-- AI Services
       |                    |    |-- Gemini 3 Flash (OCR, Text, Recipe Extraction)
       |                    |    |-- Gemini 2.5 Flash/Imagen (Image Generation)
       |                    |    |-- Gemini Live (Real-time Voice with Tool Calling)
       |                    |
       |                    |-- Tool Calling System (13 Chef Tools)
       |                    |    |-- getCurrentContext, listRecipes, checkLibraryForRecipe
       |                    |    |-- getRecipeDetails, searchRecipes, navigate
       |                    |    |-- updateShoppingList, completeShopping, updatePantry
       |                    |    |-- setTimer, setCookingMode, manageCookingStep
       |                    |    |-- generateVisuals
       |                    |
       |                    |-- Tool Registry (Server-side execution)
       |
       |-- WebSocket --> [Gemini Live Audio API]
       |                    |
       |                    |-- Tool Calls executed on backend via Tool Registry
       |                    |-- Results sent back to Gemini -> Audio response
       |
       |-- Direct Upload/Download --> [Cloudflare R2 (via Worker)]
       |
       |-- Auth + Data --> [Supabase PostgreSQL (RLS)]
       |
       |-- Subscriptions --> [RevenueCat]

Mobile Layer (React Native + Expo 54)

  • 5-tab bottom navigation: Extract, Recipes, Kitchen, Chef, Profile
  • Zustand for state management
  • React Query for data fetching
  • RevenueCat SDK for subscriptions
  • React Native Reanimated for smooth animations
  • FlashList for high-performance lists

Backend Layer (Node.js/Express)

  • REST endpoints for recipe CRUD, scan processing, URL imports, image generation
  • WebSocket server for Gemini Live Audio real-time voice
  • Deployed on Google Cloud Run

Database (Supabase PostgreSQL)

  • Tables: recipes, user_items, user_profiles, revenuecat_events, user_usage, subscription_tiers, feedback
  • Row Level Security (RLS) enabled on all tables
  • 45+ migrations for schema evolution

Gemini Integration

Feature Model Status
Recipe Extraction (URL) gemini-3-flash-preview ✅ Implemented
Image Analysis gemini-3-flash-preview ✅ Implemented
Recipe Generation gemini-3-flash-preview ✅ Implemented
Image Generation gemini-2.5-flash-image ✅ Implemented
Voice Assistant gemini-2.5-flash-native-audio-preview-12-2025 🔄 In Development
Nutrition Analysis gemini-3-flash-preview ✅ Implemented

Project Structure

CookShelf/
├── mobile/                    # React Native + Expo 54 app
│   ├── app/                  # Expo Router screens (5 tabs)
│   ├── src/
│   │   ├── components/      # Scanner, RecipeLibrary, ChefDiscovery, KitchenManager, etc.
│   │   ├── hooks/           # useSuggestions, useIngredients, useGroceryList, etc.
│   │   ├── services/        # API services, purchase service
│   │   ├── store/          # Zustand state management
│   │   └── lib/            # Utilities, audio, supabase client
│   └── package.json
├── backend/                   # Node.js/Express API
│   ├── src/
│   │   ├── routes/          # API endpoints
│   │   ├── services/        # Gemini services, AI processing
│   │   └── middleware/     # Auth, validation, usage tracking
│   └── package.json
├── supabase/                 # Database migrations (45+)
├── packages/                 # Shared TypeScript contracts
└── docs/                    # Documentation

Developer

Pol Cayuela — Full-Stack AI Developer, Catalonia, Spain

  • Self-taught full-stack developer specializing in AI integration
  • React, TypeScript, and Gemini API expertise
  • Production-ready end-to-end AI product development and deployment

What's Implemented

  • ✅ Full recipe management (create, read, update, delete)
  • ✅ Multi-source recipe import (camera scan, URL, web search)
  • ✅ Smart grocery lists with aisle grouping and ingredient merging
  • ✅ Pantry management with categories
  • ✅ Chef Discovery - AI recipe suggestions based on pantry
  • ✅ RevenueCat subscription integration with usage tracking
  • ✅ Production-ready UI with animations
  • ✅ Supabase backend with RLS security
  • 🔄 Voice assistant (Gemini Live Audio) - in development

Why I joined this hackathon: I wanted to push the boundaries of what's possible with Gemini's latest multimodal capabilities — specifically the Native Audio API and the new vision and image generation models — in a real, production-quality mobile app. CulinarIA is not a prototype: it's a fully architected product with a real monetization strategy, built to actually ship and serve users. The Shipyard hackathon was the perfect forcing function to bring everything together: AI, mobile, subscriptions, and a real audience problem, all in one submission.

Built With

Share this project:

Updates