Ask Less, Find More. Our optimization produces results in 3 turns on average, that's 7.8 times over baseline.
3 turns to the right product. 50,000 items, no LLM needed
Icon: 3 turns to the right product out of 50,000.

Ten-Turn Shopper

Ask less, find more — 2.4 turns to the right product out of 50,000.

Inspiration

Shopping online shouldn't feel like a keyword guessing game. When you walk into a store, a good salesperson doesn't make you fill out a form — they ask "what are you looking for?", listen, then ask one or two follow-ups before pointing you to the right shelf. We wanted to build an agent that works the same way: listen broadly first, narrow quickly, and surface the right product in as few exchanges as possible.

What it does

Ten-Turn Shopper is a conversational shopping agent that finds a customer's target product from a 50,000-item clothing catalog in an average of 2.4 turns — without using an LLM. Given a customer's opening message ("I'm looking for basketball shorts"), the agent asks targeted follow-up questions, accumulates preferences across the conversation, and returns ranked recommendations that improve with each exchange.

It handles four real shopping scenarios:

Buying (customer knows what they want) — 96% success, avg 1.7 turns
Browsing (customer is exploring) — 97.5% success, avg 2.4 turns
Intent Override (customer changes their mind mid-conversation) — 100% success, avg 3.7 turns
Boundary (customer has no preference for an asked attribute) — 100% success, avg 3.3 turns

Overall: 97.5% hit rate, TechnicalScore 0.874 (8.2x over the provided baseline).

How we built it

We started with the weak BM25 starter (12.5% hit rate) and identified two critical gaps: the baseline was stateless (threw away all prior context each turn) and never asked clarifying questions (wasting every turn).

Our approach is a 5-module pipeline:

NLU — Parses customer messages into structured constraints using a two-tier system. Strict regex patterns handle known customer templates with 100% accuracy; a paraphrase-tolerant fallback layer adds robustness for unseen phrasing. Includes negation detection — "no polyester" won't be misread as a polyester preference. Generates personalised messages using the customer's profile and accumulated constraints.
Conversation State — Tracks constraints across turns, detects buying vs. browsing intent, and decides what to ask next. The key insight: open-ended questions ("what matters most?") extract 2x more information per turn than specific attribute questions ("what material?"), because customers reveal their strongest preferences first regardless of category.
Clause-Aware Retrieval — Each customer constraint is a separate search clause weighted by inverse document frequency. "Triple Moon Pentagram" (1 matching product) outscores "cotton" (5,000 products). Products satisfying all clauses rank above partial matches. After intent overrides, the product category is promoted to a weighted clause to anchor results. When several products satisfy every disclosed clause equally — common for sibling colorways or sizes of the same listing — we break the tie using catalog review count (log-compressed so a handful of viral outliers can't dominate), gated strictly behind full clause coverage so popularity never outranks relevance. Together, this pushes the exact target to rank 1 in 60% of successful sessions.
Validation & Fallback — Schema enforcement, ASIN deduplication, and a 3-level exception chain. The agent never crashes, even on malformed input.
Integration — Clean module interfaces (Constraint, SessionState, NLUResult, RetrievalResult, StrategyDecision) let each component be developed and tested independently. 51 automated tests cover evaluator logic, catalog indexing, NLU parsing, constraint handling, retrieval ranking, and message phrasing.

Challenges we ran into

Ask strategy matters more than retrieval quality. Our biggest early score jump (12.5% → 85.5%) came not from improving search, but from changing what the agent asks. Asking "budget" first (0% of constraints in the dataset) vs. open-ended probing (reveals any 2 constraints) was the difference between wasting 2 turns and extracting all preferences by turn 3. Later refinements to clause coverage and ranking pushed the final score to 97.5% hit rate / 0.874 TechnicalScore.
Intent override is tricky. When a customer says "actually, forget leather — I want cotton", naively clearing all prior constraints throws away useful context (their color and size preferences are still valid). We built a compatibility check that preserves constraints unless they directly contradict the new intent, and promote the product category as an anchor to prevent generic results.
Near-duplicates look identical to BM25. Sibling colorways and sizes of the same listing frequently satisfy every disclosed clause identically, leaving BM25's field-length overlap to break ties almost arbitrarily. We added a review-count-based tie-break — but only once every disclosed constraint is already fully satisfied, so an unrelated popular product can never outrank a correct, less-reviewed one.
BM25 has a hard ceiling. 5 sessions (2.5%) are products that keyword matching still cannot distinguish from near-identical items even after tie-breaking. Dense retrieval (embeddings) would help but adds a dependency we wanted to avoid for offline scoring.

What we learned

Information-theoretic question design beats better search. The single most impactful decision was asking the right questions in the right order — not improving the retrieval algorithm. Open-ended probing first, targeted narrowing second.
Constraints are structured data, not free text. Treating each customer statement as an independent search clause with IDF weighting was far more effective than flattening everything into one OR query.
Zero-dependency systems are surprisingly competitive. Python's stdlib (sqlite3 FTS5) gets you to 0.874 TechnicalScore. The marginal gain from adding embeddings or an LLM is real but small compared to the strategy improvements — the codebase even includes an optional, off-by-default Claude-assisted parsing tier for future flexibility, but every result above comes from the stdlib-only path.

What's next

Dense retrieval via sentence-transformers to close the remaining 2.5% BM25 miss ceiling
Recommendation explanations — telling the customer exactly which detail made a product a good match, like the material or the price.
Profile-weighted retrieval — using the customer's preference tags to boost search weights, not just as a fallback signal

Built with

Python 3.10+ (standard library only — sqlite3, json, re, math)
SQLite FTS5 for BM25 full-text search with clause-aware ranking and review-count tie-breaking
Claude Code for AI-assisted development
Amazon Reviews 2023 dataset (Clothing, Shoes & Jewelry — 50K products, frozen by organizer)