PantryPilot

Inspiration

Most grocery tools optimize one product at a time — but real shopping means juggling a budget, dietary preferences, allergies, pantry items you already own, and substitutions when something is too expensive or unsafe. And AI agents? They can't reliably drive a grocery website at all: layouts shift, checkouts change, captchas appear. An agent guessing at buttons and dropdowns is a broken experience.

The WebMCP Challenge inspired us to invert that model. Instead of an agent scraping a messy DOM, what if the app handed the agent explicit, typed, discoverable tools — with the hard parts (budgets, allergies, approvals) encoded directly in the tools themselves?

What it does

PantryPilot is an agent-native grocery planning workspace. You set your constraints — budget, diet, allergies, pantry items, preferred stores — and an AI agent plans your meals, builds the grocery list, finds cheaper and safer substitutions, optimizes the basket, and validates safety. Then you review everything and explicitly approve the final basket. No payment is ever processed.

In the built-in demo: 2 adults, $85/week, vegetarian, peanut allergy, pantry with rice, garlic, olive oil and black beans. The agent detects a peanut-containing product and removes it, finds a peanut-free alternative, optimizes under budget, and shows the savings — then stops and asks for human approval.

How we built it

We built PantryPilot with Next.js 14, TypeScript, and Tailwind CSS — no backend, no API keys, 30+ seeded products. The heart of the project is 9 genuinely functional WebMCP tools registered on document.modelContext:

document.modelContext.registerTool({
  name: "search_products",
  description: "Search safe and affordable grocery products",
  inputSchema: {
    type: "object",
    properties: {
      query: { type: "string" },
      dietaryTags: { type: "array", items: { type: "string" } },
      excludeAllergens: { type: "array", items: { type: "string" } },
      maxPrice: { type: "number" },
      store: { type: "string" },
      limit: { type: "number" }
    }
  },
  execute: async (input) => searchProducts(input)
});

search_products · get_product_details · create_meal_plan · build_grocery_list · optimize_basket · suggest_substitutions · validate_basket · request_human_approval · save_user_preferences

Basket optimization is a constrained minimization — keep total spend within budget $B$ while excluding every allergen in set $A$:

$$\min \sum_i p_i q_i \quad \text{s.t.} \quad \sum_i p_i q_i \le B, \quad \text{allergens}(i) \cap A = \emptyset$$

The critical architectural choice: when no WebMCP runtime is present (Firefox, Safari, Edge), the UI falls back to a local simulator that calls the exact same functions — one codebase, two pathways, so the demo works in every browser.

Challenges we ran into

• Tool schema design — JSON Schemas are the agent's entire world. Every filter the agent needs (allergen exclusions, budget caps, stores) had to be first-class in the schema, not inferred.
• Strict validation — validate_basket must catch allergy, dietary, budget, and stock issues and report them as structured checks, warnings, and errors rather than silent failures.
• Human approval as a hard gate — making the agent unable to finish without calling request_human_approval was a deliberate design decision, not a UI courtesy.
• Graceful degradation — keeping the simulator and the WebMCP pathway on the same functions took discipline, but it's what makes the app portable.

Accomplishments that we're proud of

A live agent timeline and WebMCP tool inspector make every agent action visible. The full demo runs end-to-end in under three minutes: profile → search → allergy catch → plan → optimize → validate → approve.

What we learned

• Schema design is API design for agents — predictable inputs/outputs beat cleverness.
• Transparency (timeline + inspector) is what makes humans comfortable delegating.
• Fallbacks aren't a nice-to-have; they're what makes a demo survive any machine.

What's next

Real retailer integrations, persistent accounts, multi-store optimization, and richer dietary models — all through the same tool interface.

PantryPilot is not an autonomous checkout bot. It's a collaboration layer where agents handle the legwork and humans make the decisions.

Built With

• css
• css3
• html5
• javascript
• json
• localstorage
• next.js
• node.js
• npm
• react
• schema
• tailwind
• typescript
• webmcp