My Chefsito

Cook with confidence, starting with what you have.

Comment

Inspiration

"What's for dinner?" — it's the question we all dread. Studies show the average person spends 30+ minutes daily deciding what to cook. I built My Chefsito to eliminate that friction.

The inspiration came from a simple frustration: standing in front of an open fridge, knowing there's something I could make, but drawing a blank. Traditional recipe apps require you to search by dish name — but what if you don't know what you want? What if you just want to use what you already have?

My Chefsito flips the script: start with your ingredients, end with a delicious meal.

What it does

My Chefsito is an AI-powered cooking companion that covers the full journey — from ingredient discovery to plating.

  1. Scan Your Kitchen — Take up to 3 photos of your fridge, pantry, or countertop. GPT-4o Vision identifies ingredients and matches them against verified recipes, showing match percentages and highlighting what's missing.

  2. Chat Your Ingredients — Prefer a conversational approach? Tell My Chefsito what you have in natural language — "I've got some chicken, rice, and whatever vegetables are in the crisper" — and get recipe suggestions.

  3. Video to Recipe — Found an amazing cooking video on YouTube? Paste the link and My Chefsito extracts a fully structured recipe with ingredients, step-by-step instructions, difficulty, prep time, and more.

  4. Cooking Mode — A full-screen, step-by-step guided cooking experience with built-in timers parsed directly from instruction text, progress tracking, and an ingredient reference drawer.

  5. Conversational Mode — The hands-free experience. My Chefsito reads each step aloud using OpenAI's natural TTS voice, then listens for voice commands like "next," "back," "repeat," or "stop." The cook never needs to touch their phone.

  6. Instacart Integration — Missing ingredients? One tap sends them to Instacart with a nearby retailer pre-selected.

  7. Meal Planning — Schedule meals on a calendar by date and meal type, with 24-hour reminder notifications and real-time sync.

  8. Personalization — The app learns your dietary restrictions, allergies, skill level, favorite cuisines, and household size during onboarding to deliver truly personalized recommendations.

  9. Monetization — A freemium model powered by RevenueCat. Recipe search and chat are free with no caps. Scanning, video extraction, cooking mode, and Instacart are gated behind Chefsito Pro ($5.99/month or $39.99/year with a 7-day free trial).

How I built it

Tech Stack:

  • Flutter for cross-platform mobile development
  • Firebase for authentication (email, Google, Apple Sign-In), Cloud Firestore for user data, and Firebase Storage for profile photos
  • OpenAI GPT-4o Vision for ingredient recognition and video recipe extraction
  • OpenAI GPT-4o for AI-generated cooking instructions and chat-based suggestions
  • OpenAI TTS (tts-1, nova voice) for natural read-aloud in Conversational Mode
  • speech_to_text for on-device voice command recognition
  • Spoonacular API for verified recipe data with nutrition information
  • Instacart Developer Platform API for grocery shopping integration
  • RevenueCat for subscription management and entitlement checks
  • Resend API for in-app support requests

Architecture highlights:

  • Service-oriented architecture with singleton services for each major feature (AI, voice, subscriptions, recipes, Instacart, meal planning)
  • Hybrid AI approach: GPT-4o Vision for detection, Spoonacular for verified recipe data — combining the intelligence of AI with the reliability of a curated database
  • Recipe caching with 55-minute TTL and category-aware keys to minimize API costs
  • Completer-based TTS playback that blocks until audio truly finishes, preventing the common issue of speech recognition interrupting active audio
  • Partial speech result matching for near-instant voice command execution — commands are detected from partial STT results, not final results, eliminating the typical 3-5 second recognition delay
  • Firebase user identity synced with RevenueCat for cross-device entitlement tracking
  • Contextual paywall that shows which specific feature triggered the upgrade prompt

Challenges I faced

1. Conversational Mode — Making It Feel Natural The biggest challenge was creating a truly hands-free cooking experience. The initial implementation had a 3-5 second lag between saying "next" and the app responding. The root cause: the STT plugin was waiting for a finalResult with a 5-second silence buffer. I switched to matching commands against partialResults for instant detection, used ListenMode.confirmation with a 2-second timeout, and reduced pipeline delays throughout. The result is near-instant voice response.

2. TTS Audio Cutting Off Mid-Sentence Read-aloud would sometimes stop after 2-3 words, followed by error_no_match spam in the logs. After deep-diving into the just_audio player's async behavior, I discovered that play() returns immediately after starting playback, not after completion. This meant speech recognition was starting and killing the active audio. The fix was a Completer subscribed to the player's state stream, ensuring speak() truly blocks until playback reaches ProcessingState.completed.

3. Audio Engine Conflicts A native crash occurred when starting a cooking timer while TTS was playing. The cause: two different audio packages (audioplayers for timer sounds and just_audio for TTS) competing for the iOS audio session simultaneously. Resolved by removing the conflicting package and using haptic feedback for timer alerts instead.

4. Recipe Quality vs. Quantity APIs return hundreds of recipes, but users want the best options. I built a scoring system that weighs ingredient utilization, dietary compatibility, and cooking skill alignment — prioritizing the user's primary protein to avoid irrelevant suggestions.

5. Balancing Free vs. Premium Determining what to gate behind the paywall required thinking about user value. Recipe search and chat stay free to let users experience core value and build a habit. The AI-heavy features (scanning, video extraction, cooking mode) are premium, aligning cost with the API usage they generate.

What I learned

  • AI is a tool, not magic — The best results come from thoughtful prompt engineering, graceful fallbacks, and combining AI with verified data sources
  • Async audio is treacherous — Never assume play() means "finished playing." Always subscribe to completion states explicitly
  • Voice UX demands speed — Even 2 seconds of latency between a voice command and response breaks the illusion of a conversational experience. Partial result matching was the key insight
  • Personalization drives engagement — Generic recommendations feel impersonal; preference-aware suggestions feel like having a personal chef
  • RevenueCat simplifies everything — Going from zero to a working paywall with entitlement checks, offerings, and cross-device sync took remarkably little code

What's next

  • Saved Recipes & Collections — Let users bookmark favorites and organize by occasion
  • Grocery List Intelligence — Track what users buy to predict what they already have
  • Social Features — Share recipes with family, see what friends are cooking
  • Multi-language Support — Conversational Mode in Spanish and other languages
  • Android Launch — The Flutter codebase is ready; just needs platform-specific configuration

My Chefsito transforms the daily "what should I cook?" struggle into a delightful discovery experience. Because everyone deserves to feel like a chef in their own kitchen.

Built With

Share this project:

Updates