Cartwheel — Multiplayer Real-Time AI Shared Shopping

A real-time, multiplayer shared cart for households and groups where humans and their personal AI agents coordinate shopping via structured WebMCP tools with built-in human-in-the-loop checkpoints.

Comment

💡 Inspiration & The Problem

Most agentic shopping demos stop at a single user asking a chatbot to find and purchase an item. In the real world, shopping is deeply collaborative and contentious:

  • Roommates sharing household groceries (dietary restrictions, duplicate purchases).
  • Friends pooling money for a shared group gift (tracking who paid their share).
  • Partners budgeting shared furniture (clashing tastes and spending ceilings).

Why traditional agentic web scraping fails here

When multiple humans and agents interact in the same dynamic space, DOM scraping breaks immediately:

  1. Race conditions & stale DOM: Another tab or roommate adds an item while the agent is parsing the DOM.
  2. Lack of trust boundaries: Agents blindly clicking buttons can spend other people's money or delete items without authorization.
  3. No native negotiation mechanism: When an action affects someone else's stake, an agent cannot safely pause and wait for structured peer approval.

Cartwheel solves this with WebMCP: instead of brittle scraping, Cartwheel exposes typed, high-fidelity browser tools via document.modelContext.registerTool. Every action follows a propose → auto-resolve-or-approve paradigm, turning web shopping into a cooperative, multi-agent space with safe human-in-the-loop boundaries.

✨ What Cartwheel Does

Cartwheel unifies three complex shared shopping scenarios into a single collaborative room:

  1. 🥦 Household Groceries:
    • Multi-agent catalog exploration and auto-tag matching (e.g. vegan, gluten-free).
    • Duplicate-buy detection: when Sam’s agent tries to claim milk already claimed by Alex, it automatically flags as a conflict and requests approval.
  2. 🎁 Pooled Group Gifts:
    • Real-time contribution tracking.
    • Self-scoped mark_paid actions for tracking individual payment status.
  3. 🛋️ Shared Furniture & Big-Ticket Items:
    • Personal budget ceilings (e.g., Alex sets a $50 cap).
    • If Sam's agent proposes an item pushing total spend above Alex’s budget cap, the action blocks as pending until Alex approves or rejects with feedback.

🔄 Core Innovation: The "Propose → Auto-Resolve or Approve" Engine

Every cart mutation—whether initiated by a user click or an AI tool call—is treated as an auditable Proposal:

flowchart TD
    A[Agent or Human Action] --> B[propose_change]
    B --> C{Affects others?}
    C -- "No (Self-Scoped / Claim / Mark Paid)" --> D[Status: auto_approved]
    D --> E[Postgres apply_proposal RPC]
    C -- "Yes (Claim Conflict / Over Budget)" --> F[Status: pending]
    F --> G[Affected Participant Inbox Alert]
    G --> H{Peer Response}
    H -- "Approved" --> E
    H -- "Rejected with Note" --> I[Agent Receives Reason & Adapts]
    E --> J[Supabase Realtime Broadcast to All Tabs]
  • Self-Scoped Actions: Instantly auto-resolve and execute via database RPC, logging to the transparent activity trail.
  • Cross-Stake Actions: Automatically detect affected participants, hold the proposal in pending state, and light up the peer's inbox in real-time.
  • Adaptive Agent Feedback: When an action is rejected with a note (e.g., "Too expensive, find something under $50"), the agent reads the structured rejection note and adapts its strategy.

🛠️ WebMCP Tool Surface

Cartwheel registers six first-class tools directly onto document.modelContext:

Tool Mode Description
search_catalog readOnly Search the catalog by keyword and category with structured JSON schemas.
get_room_state readOnly Full snapshot of items, claimant status, participant budgets, and pending proposals.
propose_change mutation Add/remove/claim/swap items or update budget. Returns immediate proposal status (auto_approved or pending).
list_pending_proposals readOnly Filter proposals awaiting the user's specific approval.
respond_to_proposal mutation Approve or reject pending proposals with an optional feedback note. Prompts agent to check with human.
mark_paid mutation Instant self-scoped status update for group pooling.

🏗️ Architecture & How We Built It

  • Frontend: Next.js 14 (App Router), React, Tailwind CSS, TypeScript.
  • WebMCP Integration: Spec-compliant document.modelContext.registerTool lifecycle management with AbortController signals.
  • Backend & State: Supabase (PostgreSQL), Next.js API route handlers.
  • Single Source of Truth: apply_proposal PostgreSQL RPC function ensures atomicity. Direct table writes are blocked.
  • Real-Time Synchronization: Supabase Realtime (postgres_changes) combined with a 5-second polling fallback guarantees zero stale state across client windows and agent contexts.

🧗 Challenges We Ran Into

  1. Enforcing Trust Boundaries in Multi-Agent Systems: Allowing agents to directly mutate shared state led to authorization bypasses. We solved this by routing every action through server-verified session tokens (x-session-token), calculating affected_participant_id strictly on the server.
  2. Handling Non-Blocking Real-Time Feedback for LLMs: LLMs need immediate deterministic responses from tool calls. When an action requires peer approval, propose_change immediately returns { status: "pending", affected_participant: "Alex" }, allowing the agent to inform its human without hanging the session.
  3. WebMCP Protocol Integration & Lifecycle: Ensuring tools properly register and clean up on component mount/unmount using AbortSignal across dynamic Next.js navigation.

🏆 Accomplishments That We're Proud Of

  • True Multi-Agent Human-in-the-Loop Workflow: Built a working prototype where two distinct AI agents, serving two different humans, coordinate over shared state without conflicting or overwriting each other.
  • Zero-Click Live Synchronization: Complete end-to-end flow executing purely through WebMCP tool calls, broadcasting changes to other connected browser tabs with sub-second latency.
  • Audit-First State Architecture: Every action (even auto-approved ones) is recorded in an immutable proposal history, creating full transparency.

📖 What We Learned

  • WebMCP transforms web apps from scrape-targets into native agent collaborators. Structured schemas make agents 10x faster and eliminate brittle visual parsers.
  • Multiplayer is the ultimate test for agent safety. Trust boundaries, permission scoping, and human approval checkpoints must be baked into the data layer, not just the prompt.

🚀 What's Next for Cartwheel

  • Agent-to-Agent Direct Negotiation: Letting agents negotiate substitutions directly (e.g. proposing organic alternatives when an item is out of stock).
  • Multi-Store Checkout Automation: Hooking approved cart states into real delivery provider APIs (Instacart, Amazon, Target).
  • Split-Payment Settlement: Automated lightning network or Stripe split-payment triggers upon cart completion.

Built With

Share this project:

Updates