Shopping Copilot

Inspiration

Traditional e-commerce search often works best when shoppers already know the right keywords. Real shoppers browse, refine preferences, reject attributes, and change their minds. We built Shopping Copilot to understand that evolving intent and reach the right product in fewer turns.

What it does

Shopping Copilot is a multi-turn conversational shopping agent for the provided 50,000-product Amazon catalog. Users can:

Add preferences over time
Replace or remove earlier choices
Specify exclusions such as “no leather”
Switch categories mid-conversation
Browse vaguely or make a specific purchase request

The agent tracks category, material, color, size, style, budget, use case, features, exclusions, and previously shown products. It returns ranked recommendations and can ask a broad clarifying question to collect additional preferences.

How we built it

The Python backend implements the official Agent.reset() and Agent.respond() interface and runs entirely in memory, and makes zero external model calls.

                           Customer Message
                                  ↓
                ┌─────────────────┴─────────────────┐
                ↓                                   ↓
        Recognized Message                  Unfamiliar Free Form
                ↓                                   ↓
       Deterministic Parser            Catalog-Grounded Extractor
                └─────────────────┬─────────────────┘
                                  ↓
                         Typed Multi-Turn State
                    (Add • Replace • Remove • Exclude)
                                  ↓
               ┌──────────────────┼──────────────────┐
               ↓                  ↓                  ↓
         Weighted BM25       Exact-Facet       Category-Phrase
         Text Retrieval        Lookup            Resolution
               └──────────────────┼──────────────────┘
                                  ↓
                         Candidate Pool Fusion
                                  ↓
                      Constraint-Aware Ranking
               (Exclusions • Exact Evidence • Category Fit)
                                  ↓
                   Bounded Popularity + Lexical Tie-Breaking
                                  ↓
                      Recommendations + Clarification

The system combines:

Deterministic parsing for released simulator patterns, with a catalog-grounded fallback for unfamiliar free-form requests
Typed state for additions, replacements, removals, and exclusions
Weighted in-memory SQLite FTS5 BM25
Exact catalog-facet lookup
Phrase-level matching against the catalog category tree
Constraint-aware ranking that keeps exclusions and explicit shopper evidence ahead of popularity tie-breaking
A bounded review-volume popularity prior for otherwise indistinguishable products
An optional lightweight learned top-50 reranker for experimentation

The category resolver treats the shopper’s requested category as a complete catalog phrase instead of scattering it into unrelated keywords. When multiple products satisfy the same constraints and category, review volume is used alongside lexical rank as a bounded popularity tie-breaker.

We also built a Next.js frontend connected to the Python backend through a local API bridge, allowing the demo to show live catalog recommendations rather than static results.

Challenges we ran into

The hardest challenge was balancing sophisticated methods with end-to-end performance. Dense retrieval, semantic reranking, adaptive questions, and several Top-K strategies did not consistently improve the final score.

Handling changing intent was another challenge. A request such as:

“Blue instead of black, but still waterproof.”

must replace only the color while preserving the waterproof requirement. We addressed this with typed slots and operation-aware state transitions.

Many products also share nearly identical metadata. We resolved these large ties using phrase-level category evidence followed by a bounded review-volume prior, while keeping explicit shopper constraints authoritative.

Accomplishments that we're proud of

On the released 200-session public set:

Metric	Result
Hit Rate@10	1.000
MRR	1.000000
Mean Turns to Conversion	1.995
TechnicalScore	0.98010

The released BM25 starter achieved Hit@10 0.125, MRR 0.068034, and MTTC 9.81.

Across all 200 released public sessions, the target product appears in the top ten and is ranked first at conversion. The remaining public-score gap comes from conversational turns rather than product coverage or recommendation position.

On our team-created deterministic paraphrase stress test, the default pipeline achieved Hit Rate@10 of 1.000, MRR of 0.9975, and a TechnicalScore of 0.97315. We report this separately because it is not an organizer-provided evaluation set.

The result does not use hardcoded sample IDs, target-specific ASIN rules, catalog mutations, or evaluator changes. It comes from catalog-wide mechanisms available for every product:

Structured multi-turn state
Exact constraint matching
Phrase-level category resolution
Review-volume and lexical tie-breaking

The default pipeline requires no hosted LLM, API key, external vector database, or network access.

We also added product-disjoint synthetic evaluation, deterministic paraphrase stress testing, independent free-form extraction tests, and detailed failure-analysis tools. Public results and free-form robustness results are reported separately because they test different conversation styles and target distributions.

What we learned

Conversational shopping depends on memory and state management as much as retrieval. The system must understand what changed, what remains valid, and when another question is useful.

We also learned that greater complexity does not guarantee better performance. Several heavier semantic approaches underperformed simpler catalog-aware methods when evaluated end to end.

Most importantly, we learned to treat experimentation as part of the system. We kept changes only when they passed controlled evaluation and documented approaches that did not generalize.

What's next for Shopping Copilot

Next, we would:

Improve ranking robustness for unfamiliar free-form requests
Expand human-written dialogue evaluation
Strengthen numeric price filtering, support more complex budget expressions, and improve category-specific size handling
Support richer comparison feedback
Add recommendation explanations

Shopping Copilot demonstrates that a lightweight, offline agent can achieve strong conversational recommendation performance on the released evaluation set without hosted LLM access or an external vector database.