PantryPilot

• 

  PantryPilot — a WebMCP cooking copilot that keeps every agent action visible and human-controlled.

• 

  Planner view: PantryPilot turns pantry ingredients into a time- and diet-aware dinner plan with Agent Trace.

• 

  Human approval boundary: missing groceries wait for review; no tool can purchase or submit payment.

• 

  Guided cooking mode combines recipe steps, substitutions, Agent Trace, and a persistent timer.

• 

  Responsive finish: PantryPilot confirms dinner is ready after the guided cooking flow is complete.

Inspiration

Dinner is rarely blocked by a lack of recipes. It is blocked by a chain of small decisions: what is already in the kitchen, what fits the time and diet, what can be substituted, what is missing, and what to do next.

We wanted to turn that fragmented process into one clear, collaborative flow where an agent can help without becoming a black box.

What we built

PantryPilot is a WebMCP-enabled cooking app that turns pantry ingredients into a complete dinner plan and guided cooking session.

The agent can read the kitchen, select and scale a recipe, replace ingredients, prepare a shopping list, start cooking mode, and set persistent timers. Every action updates the same Zustand state used by the interface and appears in the visible Agent Trace.

The Golden Demo plans a vegetarian dinner for two, prioritizes tomatoes, replaces dairy with oat milk, identifies missing ingredients, pauses for human approval, and then starts the cooking flow.

How we built it

The app uses React, TypeScript, Vite, Zustand, Zod, and the WebMCP imperative API. Zod validates tool inputs, while Zustand provides one persisted state layer shared by human interactions and agent tools.

We exposed ten structured WebMCP tools through document.modelContext.registerTool. This lets agents operate named actions with validated inputs instead of guessing from pixels.

Challenges and what we learned

The hardest part was making the agent experience reliable, deterministic, and understandable. We needed every tool action to update the visible UI correctly, survive reloads, fit documented input and output budgets, and behave consistently during the Golden Demo.

The most important design decision was the human-approval boundary. The agent may prepare missing groceries, but it cannot approve, purchase, or submit payment. This keeps the experience useful while preserving user control at consequential moments.

PantryPilot taught us that capable agents need equally strong visibility, validation, and boundaries. WebMCP makes the app directly operable by an agent, while the interface keeps the person informed and in control.

Built With

• chatgpt-sites
• github
• react
• typescript
• vite
• webmcp
• zod
• zustand