pantry pal

The Alexa+ kitchen agent that cooks your food before it goes bad.

Comment

Inspiration

Food goes bad because nobody remembers what's in the fridge, and pantry apps fail because typing in every item is a chore. Alexa+ is already in the kitchen, so it should be the one that remembers.

What It Does

Pantry Pal is a self-hosted MCP server (MCP spec 2025-11-25, Streamable HTTP) that gives Alexa+ a memory of your kitchen and an agent that acts on it:

  • Snap your fridge: Show it a photo of your fridge, shelf, or grocery receipt. Amazon Bedrock (Nova vision) identifies the food and estimates expiry dates, so there's no typing.
  • Household memory that learns: Family size, diet, allergies, and dislikes persist across sessions. Say "the kids loved that" or "too spicy," and future suggestions adapt.
  • Zero-waste recipes: "What can I cook tonight?" returns recipes that use the soonest-expiring food first. An allergen safety net drops any recipe that mentions an allergy, even if the model slips.
  • Agentic weekly plan and checkout: "Plan dinners for the next 5 days" builds a plan that uses up expiring food first and reuses bought ingredients across days, then fills the shopping list. "Order it" prices the list and asks for confirmation. Only an explicit "yes" places the (simulated) order. "My groceries arrived" stocks the pantry.
  • Echo Show-style simulated Alexa+ app: Voice input, spoken replies, recipe and meal-plan carousels, an order receipt with a Place order button, and a live panel showing every MCP tool call.

How It Works

  • MCP server (src/server.ts, src/mcp.ts): TypeScript MCP SDK, StreamableHTTPServerTransport in stateless mode, Origin validation (DNS-rebinding protection from the spec), optional bearer-token authentication, and 17 tools with read-only and open-world annotations plus server instructions.
  • Amazon Bedrock (src/bedrock.ts): The Converse API with Amazon Nova Lite for recipes, multi-day plans, and image understanding. Model JSON is parsed defensively: code fences are stripped, cut-off answers are repaired, and a failed parse is retried once.
  • Agent (src/agent.ts): A Bedrock Converse tool-use loop over the MCP client. MCP tool schemas map 1:1 to Bedrock toolSpecs, and the model decides which tools to chain, in up to 8 steps.
  • Web app (src/web.ts, public/): Voice through the Web Speech API, Amazon Polly neural TTS with a natural browser-voice fallback, and all model text rendered safely with textContent.
  • Safety: Photos are read only from a sandboxed folder, with size and format checks, consent-gated checkout, 30-minute quote expiry, and no real payments.

Built During the Hackathon

Everything was built from scratch during the submission window, starting October 17, 2026.

What's Next

Connect to a real Alexa+ preview endpoint, real grocery partners for purchasing, proactive notifications such as "your spinach expires tomorrow," and hosting on Bedrock AgentCore Runtime.

Built With

Share this project:

Updates

Submission history