Counterask

Every WebMCP storefront exposes the same shape of tool: get-products, filter-products, show-products. The spec's own e-commerce example does exactly that. All of them share one assumption — that the store's job is to answer.

For most of a shopping conversation that assumption is wrong. Ask our catalog for "a belt" and 106 products match. Every one of them is a belt. The top-ranked item beats the tenth by a rounding error. Ranking them is a coin flip dressed up as a recommendation.

The store knows this. It has the distribution in front of it — it can see that one more fact, what the belt is made of, would clear about forty-seven of them. But a tool that can only return products has no way to say so. So it returns its coin flip, and the agent presents a guess as an answer.

We gave the store a second move.Counterask is a menswear storefront on 9,901 real products from the Amazon Reviews 2023 catalog. Its search_products tool returns one of two things.

When there is enough evidence, it answers — and says what still separates the products it is showing, so the agent can summarise instead of reading out a list.

When answering would be a guess, it returns status: "need_more_evidence" with the one question worth asking, the options with their counts, and why: "106 candidates, leader only 5% ahead. Asking 'material' clears ~47 of them on average (recorded on 89%)." The question is the store declining to guess on the shopper's behalf.

It reads whole sentences, not keywords: "I'm looking for a leather belt, nothing with a snap, not over $50" becomes a requirement, a refusal and a ceiling. It walks the category tree one level at a time: "shoes" → athletic → running → what for → 58 candidates. It handles a change of mind mid-conversation — "scratch that, a nylon belt" drops leather, keeps the budget, and reports what it dropped. And when nothing survives, it lifts each requirement in turn and says which one is doing the damage.

How WebMCP is used: every tool is registered through document.modelContext. answer_question exists only while a question is open — registered with an AbortSignal, removed by aborting it — so an agent reading the tool list can see what the page is waiting for. parse_only is a dry run: the agent can see how the store would read a sentence before committing. Anything the agent already knows can be passed structured and overrides the parse. A person clicking an option chip and an agent calling the tool enter the same function. Checkout is a declarative form with no toolautosubmit: the agent fills it in, the person presses it.No server, no model call, no tokens. The whole store is one page: the catalog, a thirteen-pass sentence parser, retrieval, and the stopping policy, in plain JavaScript. The engine is four parts with explicit dependencies — vocabulary, parser, retrieval, policy — and the parser is a pipeline whose order is data: each pass claims what it reads and writes a line of an audit trace that parse_only returns.

The stopping policy counts, for each attribute, how many candidates a question would clear on average, measured inside the subset that records the attribute and scaled by coverage. It asks about the best one unless the best one clears fewer than 10 candidates or under 18% of the pool. Entropy was the textbook choice and it is wrong for this data: a product carries several values of one attribute at once, so value shares are not a probability distribution.

The category tree is a facet. Every node on a product's path is a value, so "shoes" matches all shoes and "running" the running ones. A hierarchy is the one facet that can be asked again, one level at a time; ancestors the whole pool shares are never offered; depth is measured per product relative to what the shopper named.

npm test runs 75 hand-written parser cases, 4,000 fuzzed sentences, 88 assertions against the WebMCP surface, the page against three spec-shaped native implementations, a test that an open question survives a page navigation, the bundled single file, and two benchmarks. A hostile product is inserted during the tests and the suite asserts its title never lands in an instruction-shaped field.Real data broke the thing synthetic data could not. On a synthetic catalog a belt's title says "Belt". Half of real titles do not name their category — "Columbia Men's Thistletown Park Crew" is a T-shirt. Requiring the shopper's noun in the title dropped the right product in 47% of benchmark cases. Making the category tree a facet took that to 4.6%; treating a value as satisfied wherever the record carries it, and letting stray words rank rather than filter once the noun is understood, took it to 0.6%.

Missing is not a mismatch — in both directions. Only 8% of the catalog records a fit, and a naive gain calculation adores fit because asking it "removes 92% of candidates" — for having no data. Reduction is measured inside the covered subset, and an attribute recorded on under 45% of the pool is never asked about.

A fuzzer found a bug we would never have found by hand: answering a question could make the pool grow. The title-term filter fell back to no filtering when fewer than four products matched; a narrowing answer could drop the strict set below four, and the shopper watched results grow because they had answered. The reading is now decided once and frozen.

We built two-step lookahead and it was worse — selection used a two-step score while the stopping rule used a one-step score. We built the correctly formulated sequential policy and measured it at equal questions asked: the same curve. It ships off.Measured on 800 sentences generated from real product records, against a keyword matcher on identical sentences, with spread over 8 shards:

refusals inverted into requirements: 597 → 0
refused value shown in top 10: 368 → 0
budget broken in top 10: 22 → 0
Hit@10: 0.04 → 0.33 ± 0.01

On sessions that start where shoppers start — "shoes", "clothing" — walking the category tree takes Hit@10 from 0.16 to 0.31 and the final pool from 597 to 121, for half a question more. Eight shards out of eight. It is the largest single effect in the project, and it was invisible until the benchmark started at the top of the tree instead of the leaf.

We are equally proud of what we refused to do. Lowering the ask threshold from 10 to 5 buys +0.07 Hit@10 for 0.35 more questions per search, and the benchmark says take it. We didn't: the simulated shopper never gets tired, so the benchmark can price the benefit of a question and not its cost. Instead we made the cost an explicit assumption — the shipped threshold implies that roughly one shopper in four walks away each time the store asks — and published the whole frontier so the choice is visible rather than buried in a constant.The spec lists fully autonomous workflows under Non-Goals and says the human interface remains primary, with agent tools augmenting rather than replacing it. A tool that returns a question is the most direct expression of that we could build. It puts the person back in the loop at the exact moment the machine cannot decide — not as a permission prompt bolted on afterwards, but because the store genuinely does not know and says so.

And a benchmark is only as honest as its blind spots are documented. Ours draws targets uniformly, starts at the leaf, and never gets impatient. Every one of those limits changed what we shipped, and every one is written down.Measure the one number the store cannot: how often a shopper actually walks away per question. A deployed page can log question-shown, question-answered and page-abandoned; the threshold then becomes a measured quantity instead of a stated belief.

Structured input first: in a real WebMCP session the agent has already understood the shopper, so the parser should be the fallback and parse_only the first call.

A catalog with descriptions, not just titles. Attribute coverage is the ceiling on what the store can ask; this data records material on 56% of products and occasion on 25%, and the questions are limited by what the listings say.