CartWeave

Inspiration

Every week, the average household discards over $18 worth of forgotten groceries, such as wilting greens, half-used sauces, and expiring dairy. While voice assistants have lived in kitchens for a decade, their interactions remain mostly static and transactional: setting timers, reciting generic recipes, or adding disconnected items to a shopping list. They do not know what is already in your fridge, nor can they optimize a meal plan around what will spoil first.

With the emergence of Alexa+ and the open Model Context Protocol (MCP), we saw an opportunity to transform the smart display into a true kitchen concierge. We built CartWeave to give the Echo Show 15 an active understanding of household inventory. It rescues expiring ingredients, builds balanced recipes tailored to protein and calorie goals, and automatically compares grocery prices across Amazon Fresh, Whole Foods Market, and local co-ops before you spend money.

What it does

CartWeave delivers a voice-first, interactive experience that coordinates pantry management, nutrition planning, and grocery shopping in a single conversation:

• Pantry Expiry Auditing: Scans refrigerated items, categorizes spoilage risk as urgent (under 24 hours), warning (under 48 hours), or fresh, and tallies the total salvageable dollar value (averaging around $18.45 in food waste saved).
• Macro-Optimized Meal Planning: Generates a multi-day dinner itinerary structured to consume high-urgency inventory first while hitting exact protein and calorie goals.
• Multi-Store Price Arbitrage: Identifies missing ingredients and queries real-time pricing across Amazon Fresh, Whole Foods, and local co-ops, automatically factoring in Prime savings and ingredient substitutions.
• Dynamic In-Session Voice Tweaks: Enables conversational adjustments without losing session history, such as asking Alexa to make a meal dairy-free while keeping it under 520 calories.
• Interactive Display Cards: Synchronizes Alexa voice responses with four visual cards on the Echo Show 15 screen:
  1. Pantry Expiry Grid: Visual fridge inventory with spoilage countdown indicators.
  2. Meal Plan Cards: Day-by-day recipe breakdown with nutritional macros and swap options.
  3. Store Comparison Table: Direct side-by-side cart pricing and store savings breakdown.
  4. Order Confirmation Card: Delivery booking receipt tracking net grocery waste dollars saved.

How we built it

CartWeave is engineered as a full-stack project combining protocol handling, cloud AI orchestration, and a dedicated device simulator:

• Protocol Transport Layer: Built on the official Model Context Protocol Streamable HTTP specification. We implemented session management headers across standard HTTP requests and real-time Server-Sent Events streams.
• Cloud AI Orchestration: Integrated the Amazon Bedrock Converse API to coordinate Claude 3.5 Sonnet and Amazon Nova Pro across five stateful tools:
  • Pantry Inventory Scanning: Analyzes inventory shelf life and urgency levels.
  • Meal Plan Generation: Calculates recipes against protein, calorie, and budget constraints.
  • Grocery Cart Comparison: Compares basket totals across multiple grocery retailers.
  • Ingredient Constraint Swapping: Modifies a specific meal in place while preserving cart state.
  • Smart Basket Checkout: Confirms delivery, updates inventory, and calculates savings.
• Multimodal Echo Show 15 Simulator: Built with Next.js, React, and Tailwind CSS. Features an Echo Show hardware frame, an animated voice orb, browser-native speech recognition and Alexa voice synthesis, and a live protocol inspector.
• Serverless Cloud Deployment and Offline Testing: Packaged with an AWS Serverless Application Model template for deployment to AWS Lambda, API Gateway, and DynamoDB. We also implemented a deterministic local mock mode allowing judges and developers to run the full workflow offline without AWS credentials.

Challenges we ran into

• Stateful Execution over HTTP: Adapting stateless HTTP to maintain long-lived conversational pantry sessions required custom session header plumbing and synchronizing real-time event streams alongside parallel command calls.
• Tool Schema Alignment: Ensuring dynamic parameter validation schemas mapped cleanly to Amazon Bedrock tool specifications without throwing validation errors during mid-dialogue constraint swaps.
• Voice and Screen Synchronization: Timing Alexa speech synthesis with real-time card transitions on the simulated Echo Show 15 display to prevent layout delays and voice lag.

Accomplishments that we're proud of

• Full Protocol Compliance: Built a fully compliant Model Context Protocol Streamable HTTP implementation backed by an automated testing suite.
• Zero-Cost Local Evaluation: Created an offline mock planner so judges can clone the repository and immediately test the end-to-end flow without configuring cloud keys or incurring API costs.
• Turnkey AWS Infrastructure: Designed an AWS CloudFormation template that provisions the Lambda runtime, API Gateway, and DynamoDB storage in one deployment command.
• Open Source Contribution: Documented our architectural findings and provided a transparent developer friction log to help the ecosystem build on Model Context Protocol and Alexa+.

What we learned

• Multimodal Voice and Screen Balance: How to design assistant systems where voice audio and visual cards complement each other, letting voice handle quick confirmations while high-density cards display price comparison tables and nutritional breakdowns.
• Decoupled Architecture: The architectural patterns of Model Context Protocol for decoupling AI reasoning from backend smart home tools.
• Prompt and Tool Optimization: Practical multi-turn prompt engineering and tool calling using the Amazon Bedrock Converse API.

What's next for CartWeave

• Direct Retailer Cart Integration: Connecting with live grocery ordering APIs to automatically populate external shopping carts with a single tap.
• Computer Vision Fridge Scanning: Connecting smart fridge cameras or mobile photo uploads to automatically populate expiry grids without manual entry.
• Physical Device Rollout: Deploying CartWeave onto physical Echo Show 15 devices as native Alexa+ developer access expands.

Built With

• alexa-plus
• amazon-bedrock
• amazon-nova-pro
• api-gateway
• aws-lambda
• aws-sam
• claude-3-5-sonnet
• dynamodb
• echo-show
• express.js
• mcp
• model-context-protocol
• nextjs
• node.js
• react
• tailwindcss
• typescript
• zod