Inspiration
I asked an AI assistant to find wireless headphones under $50. It returned five polished recommendations: prices, brands, pros, cons, all written with confidence. Three of them did not exist. Not “out of stock.” Completely made up.
That problem gets worse for people who cannot easily see a screen, use a mouse, or type a detailed search query. If shopping is already difficult to navigate visually, an AI assistant that invents products or silently makes decisions is not helpful; it is another barrier.
Co-Cart began with two questions:
- How can an AI agent shop without inventing what it found?
- How can someone who cannot see or type comfortably still control that shopping experience?
The answer is a shared human-and-agent shopping layer.
For a person using the screen, Co-Cart provides simple decision cards and a verified shortlist.
For a person who is blind, low-vision, has limited motor control, cannot type, or simply needs hands-free help, Co-Cart provides a voice-first path through the same experience. The agent reads choices aloud, accepts spoken answers, explains results, and prepares cart changes, but it never takes away the shopper's final decision.
For a WebMCP-enabled browser agent, Co-Cart exposes 13 structured tools. Instead of scraping pixels, guessing which button matters, or inventing product details, an agent can read the visible shopping state, answer the same questions, run the verified search flow, inspect real results, and propose safe cart actions.
The human sees the experience.
The voice user hears the experience.
The browser agent uses the experience.
All three operate on the same truth.
What it does
Co-Cart is an accessible, human-in-the-loop shopping assistant for meals, gadgets, and clothing.
When Co-Cart loads in a WebMCP-enabled browser or agent host, it silently registers 13 tools using navigator.modelContext.
This lets an agent do useful work safely:
Read the current shopping state
↓
Choose a category
↓
Answer the same visible decision cards
↓
Start the real OpenAI → Shopify search
↓
Read verified live results
↓
Highlight recommended products
↓
Propose a cart addition, removal, or swap
↓
Wait for the human to approve or reject it
### 2. A voice-first shopping flow for people who cannot see or type
The hands-free mode is a core feature, not a voice-search add-on.
A shopper can use speech to:
- Hear what screen they are currently on
- Hear each available choice read aloud
- Answer decision questions without touching a keyboard
- Start the live Shopify search
- Hear the recommended products, prices, merchants, and tradeoffs
- Ask the agent to repeat information
- Create cart proposals safely
- Approve or reject important actions with explicit spoken phrases
This matters for blind and low-vision users, people with motor impairments, users with temporary injuries, and anyone unable to use a conventional screen-and-keyboard shopping flow.
The microphone remains **off by default**. It only starts after the shopper explicitly enables hands-free mode.
Most importantly, voice users are not pushed into a limited fallback experience. Voice mode uses the same live Shopify catalogue, the same verified product facts, the same WebMCP-style guarded tools, and the same human approval rules as the visual interface.
## What it does
Co-Cart is a human-in-the-loop shopping assistant for meals, gadgets, and clothing.
A shopper answers six simple questions, such as:
- What are you trying to buy?
- What matters most: best value, crowd favourite, hidden gem, or industry standard?
- What is your budget?
- Where should it arrive?
Then Co-Cart:
1. Starts a real OpenAI shopping agent.
2. Searches Shopify Global Catalogue through MCP.
3. Verifies product facts from the merchant data.
4. Shows a short, explained list of options with tradeoffs.
5. Lets the shopper add products or approve/reject agent-made cart proposals.
6. Opens the real merchant checkout only when the shopper is ready.
It also includes a hands-free voice experience for people who cannot comfortably see, point at, or type into a typical shopping interface. The voice agent can read options aloud, accept spoken answers, describe live results, and create proposals using the same guarded tools and safety rules as the visual experience.
## WebMCP: the agent action layer
WebMCP is central to Co-Cart.
WebMCP does **not** come with a fixed list of tools that every website must use. Instead, it gives a website a browser API — `navigator.modelContext` — for declaring its own safe, structured capabilities to an AI agent.
Co-Cart uses WebMCP to register **17 custom shopping tools** designed specifically for this experience.
They let a WebMCP-enabled browser agent:
- Read the current shopping state
- Choose Meals, Gadgets, or Clothing
- Answer individual decision cards
- Answer an entire shopping brief in one batch call
- Start the verified OpenAI → Shopify search
- Read live, server-verified results
- Inspect individual products
- Highlight recommended products for the shopper
- Read the cart and its separate currency totals
- Check dietary, allergen, or numeric constraints
- Save shopping preferences
- Propose cart additions, removals, and swaps
The agent does not get a hidden admin mode.
Every WebMCP tool maps to an action or state the shopper can see in the interface. The visual UI, voice mode, and WebMCP tools all use the same shared shopping state.
```text
WebMCP itself Co-Cart
───────────── ───────
Provides the browser API Defines 14 shopping-specific tools
Does not prescribe tool names Chooses only actions useful for shopping
Connects page and agent Shares state with the visible UI
## How we built it
Co-Cart is built with React, TypeScript, Vite, Zustand, Tailwind CSS, OpenAI, Shopify Global Catalogue MCP, WebMCP, and Netlify Functions.
The main architecture is:
```text
Human UI + WebMCP tools
↓
Shared Zustand state
↓
Netlify server function
↓
OpenAI Responses API agent
↓
Validated Shopify MCP calls
↓
Verified Shopify product data
↓
Shortlist, tradeoffs, and cart proposals
The most important design choice is that the agent does not directly control Shopify or write product facts. The agent can select Shopify variant IDs and explain its reasoning. The server validates every tool call, retrieves the corresponding Shopify data, and reconstructs the product title, price, seller, image, and links from the verified response.
For WebMCP, Co-Cart silently registers 13 browser tools through navigator.modelContext. A WebMCP-enabled agent can:
- Read the current shopping state
- Select a category
- Answer visible decision cards
- Start the live search
- Read verified results
- Highlight products
- Propose add/remove/swap cart changes
- Check cart constraints
The key idea is that the human UI and agent tool layer use the same state. The agent does not get a secret admin mode. If it changes a decision, the UI updates accordingly. If it proposes a cart action, the shopper still approves or rejects it.
For the demo experience, we also captured genuine live Shopify search results for three default paths. These replay in about two seconds with a visible search sequence. Any changed answer falls back to a real live search.
Challenges we ran into
Making AI recommendations trustworthy
The obvious first version was to let the model return a JSON list with names, prices, and merchants. That was risky. Even a very capable model can produce plausible but inaccurate facts. We changed the design so that the model only chooses IDs from Shopify tool output. The server rejects any selected ID that Shopify did not return.
Keeping agent autonomy safe
An agent that can search should not automatically be able to buy. Co-Cart separates confirmed cart items from agent proposals. An agent can recommend an addition, removal, or swap, but the shopper must explicitly approve it. Checkout always goes to the merchant; Co-Cart never submits payment.
Making voice genuinely accessible
We did not want voice mode to be a weaker fallback. It uses the same live-search pipeline, verified product data, cart guardrails, and approval rules as the visual interface.
The microphone also stays off until the shopper explicitly starts hands-free mode.
Accomplishments that we're proud of
- Built a real WebMCP action layer with 13 guarded browser tools.
- Made the WebMCP tools invisible in the product UI, so the experience remains simple for regular shoppers.
- Ensured every displayed product fact is verified from live Shopify data rather than generated by the model.
- Added explicit human approval for every agent-initiated cart change.
- Built an accessible voice-first flow for people who cannot see or type comfortably.
- Prevented misleading multi-currency totals by keeping each currency subtotal separate.
- Added tests for the trust boundaries: rejected model-made products, missing Shopify search calls, malformed tool arguments, stale searches, rejected proposals, and more.
- Deployed the app publicly on Netlify.
What we learned
The biggest lesson was that reliability is not the same as hiding failure.
At first, it was tempting to fall back to static demo products if the live search failed. That made the app look reliable, but it also made it impossible to tell whether results were real.
We removed silent fallbacks. If OpenAI or Shopify fails, Co-Cart says so clearly.
We also learned that WebMCP is most powerful when it is treated as infrastructure, not as a flashy feature. The agent tools should be discoverable to the browser agent, constrained by clear schemas, and connected to the same state the human sees.
Finally, accessibility is not just larger text or a microphone icon. A voice user should be able to complete the same meaningful workflow as everyone else: understand choices, make decisions, inspect tradeoffs, and remain in control.
What's next for Co-Cart
Next, I want to build:
- Stronger provenance, including timestamps and signed merchant-data claims.
- More delivery-country support and real address validation.
- Better voice accessibility features, including richer result summaries and more flexible confirmation flows.
- Merchant-facing insights that show businesses how agents discover, compare, trust, or skip their products.
The broader goal is simple: make shopping websites useful to both humans and the agents acting for them, without sacrificing truth, accessibility, or human control.
Built With
- and
- co-cart-is-built-with-react
- netlify
- openai
- shopify-global-catalogue-mcp
- tailwind-css
- typescript
- vite
- webmcp
- zustand
Log in or sign up for Devpost to join the conversation.