Sous Chef

An AI-powered kitchen assistant that converts YouTube cooking tutorials into interactive step-by-step guides with real-time vision feedback and hands-free control—no more smudged screens.

Comment

Inspiration

Our goal? A digital "Remy" from Ratatouille—an AI that isn't just a chatbot, but a seeing, hearing assistant that guides you and lets you cook.

What it does

  • Instant Recipe Parsing: Paste a YouTube URL, and it extracts ingredients and timestamped steps.
  • Hands-Free Guide: It watches your progress and listens for cues. No more touching your device while prepping.
  • Vision-Aware AI: Ask "Is this browned enough?" and the AI analyzes your live camera feed to give you a real-time answer.
  • Culinary Timeline: Can snap photos of your progress to build a personal food diary.

How we built it

  • Used Gemini 3.0 Pro for generating the recipe; it uses Google Search to verify info and turns video content into a perfectly structured, timestamped recipe.
  • For the "Guide Mode," Gemini Live acts as the eyes and ears, streaming your kitchen feed in real-time.
  • Most of the UI (React 19 & Tailwind) is Vibe-coded with AI Studio with some manual code changes.

Challenges we ran into

  • Before I figured out how the version control works on AI Studio, there were a couple of times that the response broke the app & I accidentally refreshed the page & couldn't just revert because the Restore button isn't there. All good now though!
  • Getting the Gemini API to return the correct recipe for different languages.
  • Turning a chaotic cooking vlog into a structured recipe with perfect timestamps took a lot of prompt-tuning and schema validation.

Accomplishments that we're proud of

  • Got a working prototype in a very short time.
  • That "magic" moment. The first time we pointed the camera at a bowl of chopped carrots for a step that required chopped onions, and asked, "How does this look?" and the AI said, "That's onions, not carrots."

What I learned

  • Multimodality changes everything. Text chat is fine, but an AI that can actually see your work-in-progress is a paradigm shift for productivity tools.
  • Really important to have the redetail when prompting the AI to change something.

What's next for Sous Chef

  • Build it into a mobile app
  • Integrate with AR glasses
  • Add the ability to point a phone at the fridge, and it suggests recipes

Built With

Share this project:

Updates