Kitchen Studio

Kitchen Studio - an AI kitchen copilot that cuts decision fatigue and cooks with you in real time

Comment

Inspiration

Cooking apps usually start at the wrong place.

They assume the user already knows what they want to make, has the energy to browse, and has time to read through long recipes. Real life is messier than that. Most people open a food app because they are tired, indecisive, low on time, trying to avoid wasting ingredients, or already halfway into a cooking mistake.

That inspired Kitchen Studio.

We wanted to build something that moves beyond a recipe search box and behaves more like a live kitchen agent: an assistant that helps you decide what to cook right now, based on your context, and then stays with you while you cook through voice, interruption handling, and adaptive guidance.

Our core idea was simple:

Don’t just recommend recipes. Recommend decisions.
And once the decision is made, help the user actually finish the meal.

What it does

Kitchen Studio is a live multimodal kitchen copilot.

It helps users:

  • decide what to cook based on pantry items, expiring ingredients, time, and context
  • reduce decision fatigue with a single high-confidence suggestion like Tonight’s Meal
  • start a realtime cooking session powered by Gemini Live
  • talk naturally with the agent while cooking
  • interrupt it mid-sentence, ask follow-up questions, and recover from mistakes
  • get adaptive guidance instead of static step-by-step text

Instead of making the user scroll through endless options, Kitchen Studio tries to answer:

“What should I cook right now, and help me do it.”

How we built it

We designed Kitchen Studio as two connected layers:

1. Decision layer

This layer figures out the best meal to suggest in the user’s current moment.

It uses contextual signals such as:

  • pantry state
  • expiring ingredients
  • available cooking time
  • time of day
  • user preferences
  • recent cooking behavior

We built a scoring-based ranking system to generate a strong default decision rather than dumping the user into search. That lets the app surface a focused recommendation such as a quick rescue meal, lazy dinner, or post-gym meal.

2. Live cooking layer

Once the user starts cooking, Kitchen Studio switches from recommendation mode into realtime agent mode.

This layer uses Gemini Live for natural voice interaction, while a deterministic orchestration layer manages the actual cooking session. We intentionally did not let the model directly control authoritative app state. Instead, we used a structure where the model requests actions and the app decides whether to execute them.

That architecture includes:

  • a hierarchical cooking session state machine
  • event-driven tool execution
  • timer handling
  • interruption / barge-in support
  • session event logging
  • recovery logic for confusion or low confidence
  • safety-oriented rule boundaries around critical steps

This matters because cooking is not just chat. It has timing, sequence, mistakes, interruptions, and real-world consequences.

Tech architecture

Our stack combines mobile app UX, realtime AI, orchestration, and cloud infrastructure:

  • Frontend/mobile: React Native + Expo
  • Realtime AI: Gemini Live API
  • SDK layer: Google GenAI SDK
  • Backend orchestration: Supabase Edge Functions
  • Database / persistence: Supabase Postgres
  • Cloud hosting: Google Cloud / Cloud Run components
  • Session logic: XState-based state machine orchestration
  • Tooling: timer actions, state transitions, checkpoint handling, and structured event logging

The result is a system where Gemini provides the live conversational intelligence, while the app and backend enforce the workflow, persistence, and safety boundaries.

Challenges we ran into

This project was much harder than building a normal chatbot.

1. Realtime voice is messy

A live agent has to feel natural. That means handling:

  • interruptions
  • partial user speech
  • playback buffering
  • timing between user speech and agent speech
  • resume behavior after network hiccups

The experience breaks quickly if the agent keeps talking over the user or cannot recover smoothly.

2. Cooking needs structure

A freeform model is not enough for a real cooking flow. Cooking has ordered steps, timers, checkpoints, and safety concerns. We had to design the system so the conversation feels natural without losing deterministic control of the underlying cooking state.

3. Cloud architecture tradeoffs

Long-lived live sessions are not the same as normal request-response APIs. We had to think carefully about what should live on the client, what should be persisted on the backend, and how to make the system resumable instead of assuming one perfect uninterrupted session.

4. The product problem was bigger than recipes

The real challenge was not just “generate cooking help.” It was solving the deeper UX problem of decision fatigue. That changed the whole product direction from a recipe app into a decision-first kitchen assistant.

What we learned

We learned that the best multimodal agents are not just impressive because they talk. They are impressive because they are grounded.

For Kitchen Studio, grounding came from:

  • pantry context
  • timing context
  • user state
  • deterministic workflow state
  • structured tool calls
  • explicit recovery paths

We also learned that a strong agent experience needs a clear split between:

  • conversational intelligence
  • product logic
  • state authority

That separation made the system feel more reliable, more debuggable, and more realistic as a product.

Why we think this matters

There are many apps that help people find recipes. Far fewer help people make a decision quickly and then stay with them through the act of cooking.

Kitchen Studio is our attempt to build that missing layer: a kitchen experience that is not just searchable, but assistive, contextual, and live.

We believe the future of consumer AI is not just chat windows. It is agents that understand the moment the user is in and help them move through it in real time.

Kitchen Studio brings that idea into one of the most universal daily problems:

figuring out what to eat, and actually cooking it.we ran into

What's next for Kitchen Studio

We see Kitchen Studio growing far beyond a live cooking copilot into a smarter food platform that helps users make better everyday kitchen decisions and turns cooking into something more social, more personalized, and more connected.

1. Tools for creators

We want to build creator features that let food creators do more than just post recipes.

Future versions could let creators:

  • turn recipes into interactive guided cooking flows
  • publish creator meal packs and weekly plans
  • create short-form cooking experiences for different moods or moments
  • attach voice guidance, tips, and substitutions to their recipes
  • build more immersive content that users can actually cook along with

The goal is to help creators move from static content into live, reusable cooking experiences.

2. More advanced recommendations

Right now, Kitchen Studio focuses on reducing decision fatigue with contextual meal suggestions. Next, we want recommendations to become much smarter and more adaptive.

That includes:

  • recommendations based on time, budget, energy level, pantry state, and cravings
  • better understanding of what a user actually likes versus what they save
  • smarter “use what you already have” suggestions
  • more accurate expiry-aware meal decisions
  • recommendations that balance convenience, nutrition, and food waste reduction

Over time, Kitchen Studio should feel less like a recipe browser and more like a decision engine built around your real life.

3. Store and commerce connections

A big next step is connecting Kitchen Studio with stores and grocery workflows.

That could include:

  • turning missing ingredients into smarter shopping lists
  • comparing where to buy ingredients
  • connecting users to nearby stores or delivery services
  • building meal decisions around what is actually easy to get
  • helping users restock based on cooking habits and pantry usage

This would let Kitchen Studio bridge the gap between meal intent and actually getting ingredients into the kitchen.

4. Sharing and cooking with friends

Food is social, and Kitchen Studio should reflect that.

Future social features could include:

  • sharing meal ideas and recipes with friends
  • collaborative shopping lists
  • sending “you should cook this” recommendations
  • creator and friend-based discovery
  • group meal planning for roommates, couples, or families

We want Kitchen Studio to make cooking feel less isolated and more connected.

5. Smarter kitchen memory

Another major area is memory and personalization.

As users keep using Kitchen Studio, it should better understand:

  • their favorite meals
  • what ingredients they always keep at home
  • what recipes they abandon halfway
  • which substitutions they prefer
  • how quickly they cook
  • the types of meals they make on different days

That memory can make the assistant more useful, faster, and more personal over time.

Built With

Share this project:

Updates