Shopping Copilot AI: Conversational E-Commerce Search Project Description

1. Inspiration

Traditional e-commerce search usually requires customers to enter precise keywords from the outset. In real-world scenarios, however, customers’ initial shopping needs are often incomplete. For example, a customer may only know that they “want a pair of shoes suitable for commuting in the rain” without having decided on the material, color, size, or budget. Conventional keyword search struggles to understand needs that emerge gradually, often returning a large number of relevant but insufficiently precise products and forcing customers to continue filtering. The frustration caused by repeated inaccurate recommendations can ultimately reduce their motivation to shop.

Based on this insight, we developed Shopping Copilot AI—a deterministic, explainable, and cost-efficient conversational shopping agent. Instead of making blind recommendations when information is insufficient, it asks targeted follow-up questions to progressively narrow the candidate set, balancing recommendation accuracy with interaction efficiency.

The project aims to identify and recommend up to 10 products that precisely match the customer’s target needs within 10 conversation rounds. To achieve this, it combines multi-turn state management, constraint reasoning, retrieval, ranking, and dialogue decision-making. The system is evaluated rigorously using exact ASIN matching.

2. What It Does

Shopping Copilot AI is a conversational product recommendation assistant designed for e-commerce search. It addresses the difficulty of quickly finding the right product when customers express incomplete requirements or use imprecise keywords. Customers do not need to provide every filter at the beginning. They can simply describe their general needs in natural language—such as the intended use, style preferences, budget range, or functional requirements—and the system progressively identifies their actual shopping intent through a multi-turn conversation.

During every interaction, Shopping Copilot AI performs four actions: understanding the request, updating the conversation state, deciding what to do next, and generating recommendations. The system reads an anonymized customer preference profile and the current message, then extracts key information such as product category, size, color, material, brand, budget, style, functionality, and intended use. It merges this information into a continuously maintained conversation state.

The system distinguishes among newly added requirements, revised preferences, neutral responses, and uncertain statements. This prevents contradictory recommendations that could result from simply accumulating every keyword from the conversation history.

One of the project’s highlights is its catalog-driven clarification strategy. When a customer’s requirements are still unclear, Shopping Copilot AI does not mechanically ask questions from a fixed list. Instead, it examines the current candidate products and asks about the attributes that would most effectively distinguish between them. For example, if the main differences among the candidates are size, material, or intended use, the system prioritizes those attributes. This allows it to narrow the candidate set in as few conversation rounds as possible.

During product recommendation, the system returns up to 10 product parent_asin values ranked by matching confidence. Candidate retrieval and ranking use multiple signals, including the customer’s current message, conversation history, anonymized preference profile, product title, category, attributes, usage requirements, and product popularity. The system therefore does not rely on a single keyword-matching path. Explicit hard requirements—such as category, size, and budget—are applied as constraints first, reducing the likelihood that clearly unsuitable products will enter the final recommendation list.

Shopping Copilot AI also uses confidence gating to control the number of recommendations. When information is insufficient, it returns only a small number of the most reliable results to avoid introducing noise. As the customer’s needs become clearer and the candidate set becomes smaller, the system expands its output into a more complete Top-K recommendation list. When the customer does not want to answer further questions or the available evidence is already sufficient, the system promptly returns up to 10 high-confidence products.

This design balances the decision to continue asking questions against the need to provide an immediate recommendation. It directly addresses the challenge’s central objective: identifying the products a customer truly wants within no more than 10 conversation rounds through proactive clarification, state management, constraint reasoning, and ranked recommendations.

3. How We Built It

We used a modular architecture that separates dialogue understanding, state management, retrieval, ranking, and output decisions into independent components. This makes the system easier to test, debug, maintain, and extend.

Multi-Turn State and Constraint Management

state.py and constraints.py record the preferences expressed by customers across different conversation rounds. The system distinguishes among newly added requirements, neutral responses, and preference overrides. When a customer changes their mind, the superseded constraint is correctly invalidated and no longer affects subsequent retrieval.

This explicit state-management approach makes every recommendation traceable to specific customer requirements. It also avoids preference loss and conversation inconsistencies commonly found in black-box dialogue systems.

Intent Recognition and Clarification Strategy

intent.py identifies purchasing, browsing, intent-override, and edge-case scenarios. Based on the currently known requirements and the product catalog, dialogue_policy.py selects high-value attributes that have not already been discussed. Each question can ask about up to three related attributes.

Clarification questions are driven by the current candidate products and their catalog attributes rather than by a fixed list. The system can therefore prioritize questions that genuinely help distinguish between products and reduce unnecessary conversation rounds.

Multi-Path Product Retrieval

retrieval.py builds an in-memory SQLite FTS5 full-text search index for 50,000 products. It generates candidate products through multiple complementary retrieval paths, including:

The customer’s current message;
Historical conversation state;
Product titles and categories;
Product attributes and usage requirements;
The anonymized preference profile;
Product popularity;
Relaxed-constraint and fallback retrieval.

Multi-path retrieval reduces the system’s dependence on any single query formulation. Even if one retrieval path fails to recall the target product, other paths can still contribute it to the candidate set.

Exact Catalog Signatures

We normalize product attributes and details and use them to build an exact product-signature index. When a customer expresses specific requirements that are explicitly supported by the catalog, the system intersects the product sets associated with those attributes, rapidly narrowing the candidate pool.

Ultimately, we want the system to do more than perform fuzzy matching between customer messages and product titles. It should transform the requirements revealed throughout the conversation into verifiable catalog constraints.

Hard-Constraint Filtering and Rank Fusion

Candidate products first pass through hard-constraint filters for attributes such as category, budget, and size. This prevents products that violate explicit requirements from entering the recommendation list solely because of textual similarity.

ranking.py then uses weighted Reciprocal Rank Fusion (RRF) to combine the results from different retrieval paths while applying bounded boosts to high-quality evidence. RRF does not require the raw scores produced by different retrieval methods to be directly comparable, making it well suited to combining heterogeneous signals such as full-text search, attribute matching, customer profiles, and product popularity.

Engineering Reliability and Evaluation

agent.py coordinates the complete execution flow and controls the number of recommendations according to evidence strength. local_evaluator.py simulates multi-turn customer conversations and uses strict parent_asin equality to calculate Hit@10, Mean Reciprocal Rank (MRR), Mean Turns to Correct (MTTC), token usage, and performance across different scenario types.

The project currently includes 142 runnable core unit tests covering state management, constraints, intent recognition, lexicon processing, clarification policies, retrieval, ranking, and evaluation logic. We also retained evaluation results from multiple versions, making the system’s performance improvements traceable and reproducible:

V0 baseline technical score: 0.106710
V1.1 multi-path RRF: 0.159437
V2 intent-aware dialogue: 0.342336
V2.1 evidence-aware ranking: 0.422530
V3.3 final version: 0.968075

Development Tools

The main tools used to develop and reproduce the project include:

Python 3.10 or later;
Git for version control and experiment tracking;
Python’s unittest framework for automated unit testing;
A local command-line environment for running the evaluator and reproducing experiments.

The project does not depend on a specific IDE and can be developed or run in VS Code, PyCharm, or any other environment that supports Python.

APIs, Libraries, and Frameworks

V3.3 does not call any external APIs at runtime. It also does not use OpenAI GPT-4o or any other large language model. As a result:

No API key is required;
No network service is required;
No prompt or completion token costs are incurred;
The reported prompt-token and completion-token usage are both zero.

The project uses only the Python standard library. Its core technologies include:

sqlite3 and SQLite FTS5 for building the in-memory full-text search index;
json for reading the JSONL product catalog and evaluation data;
re for rule-based intent, constraint, and attribute recognition;
unittest for automated testing;
Python’s built-in data structures and sorting tools for state management, signature indexing, candidate fusion, and result ranking.

We deliberately avoided third-party dependencies such as PyTorch, Hugging Face Transformers, scikit-learn, and pandas. This demonstrates that, in a fixed-catalog environment with strict latency and exact-match requirements, a thoughtfully engineered deterministic system can still achieve competitive results.

Datasets and Project Assets

The project uses the following datasets and assets:

data/catalog.jsonl: 50,000 products from the Clothing_Shoes_and_Jewelry category of the Amazon Reviews 2023 dataset;
data/public_set.jsonl: 200 labeled public conversations, including 80 purchasing scenarios, 80 browsing scenarios, 30 intent-override scenarios, and 10 edge-case scenarios;
artifacts/lexicon.json: a rule-based lexicon automatically generated from the product catalog;
artifacts/category_playbook.md: a catalog-driven resource used for category recognition and clarification strategies;
results/v3.3-final.json: per-conversation evaluation records for all 200 conversations in the final version;
DATA_ATTRIBUTION.md: documentation covering data sources, attribution, and permitted use.

The project does not supplement the public evaluation set with additional manually labeled data. Its main processing resources are generated automatically from the existing product catalog.

4. Challenges We Ran Into

The first challenge was achieving reliable recall under strict ASIN exact-match evaluation. With keyword search alone, a semantically similar result with a different product identifier would still be considered incorrect. We progressively addressed this issue through multi-path retrieval, hard-constraint filtering, and exact catalog signatures.

The second challenge was maintaining consistency across a multi-turn conversation. Customers may add conditions, reject earlier preferences, modify them, or withdraw them entirely. Simply accumulating all historical keywords produces conflicting constraints. We therefore designed explicit override and invalidation mechanisms so that the current state always reflects the customer’s latest intent.

The third challenge was balancing “continue asking questions” with “recommend now.” Returning too many products too early reduces ranking quality, while repeatedly asking questions increases customer effort. Confidence gating allows the system to adjust Top-K dynamically according to evidence strength, making an explainable trade-off between recommendation accuracy and interaction efficiency.

The fourth challenge was ensuring that performance improvements were verifiable. We did not tune the system based only on a few demonstration cases. Instead, we retained complete evaluation results for every version and established automated tests for the core modules, allowing us to measure the actual effects of each architectural change on Hit@10, MRR, and MTTC.

Several limitations still need to be addressed:

The final results are based on 200 public conversations and have not yet been validated against a locked private holdout set;
Rule-based language parsing has limited coverage of spelling errors, multilingual input, implicit preferences, and previously unseen synonyms;
Exact signatures depend on product metadata quality and the disclosure expressions currently supported by the system;
The in-memory index is rebuilt whenever an agent is created, and startup time and peak memory usage have not yet been systematically benchmarked;

5. Accomplishments That We’re Proud Of

Throughout development, our team designed around realistic shopping scenarios and gradually transformed a simple baseline into a complete conversational retrieval system.

From a technical perspective, we combined multi-turn state management, intent recognition, catalog-driven clarification, multi-path retrieval, exact signatures, RRF ranking, and confidence gating in a clearly structured architecture. Each module has a well-defined responsibility, can be tested independently, and can be replaced or extended in future versions.

In terms of performance, the system improved Hit@10 from 0.125 in V0 to 1.0 in V3.3, while reducing MTTC from 9.81 to 2.14 turns. Across all 200 public conversations, the target product was found by the fourth round, and 189 target products were ultimately ranked first.

From a feasibility perspective, Shopping Copilot AI requires no GPU, model downloads, API keys, or continuous network connection. Zero external API calls and zero token costs result in low operating costs, stable output, and predictable latency. The system is also unaffected by unpredictable behavioral changes caused by third-party model updates.

This architecture is useful beyond the competition. It can be extended to real-world scenarios with well-defined inventories, structured product attributes, and a need for precise recommendations, including apparel, footwear, electronics, home goods, and enterprise procurement catalogs.

6. What We Learned

First, we learned that the core of conversational search is not making the system “sound more human.” The real objective is to ensure that every conversation round meaningfully reduces uncertainty. A good clarification question should directly change the candidate set rather than simply making the interaction feel natural.

Second, we found that a structured product catalog is itself a valuable source of intelligence. Once categories, attributes, functions, and usage scenarios are properly normalized, many requirements can be addressed reliably through deterministic reasoning without necessarily calling a large language model.

We also confirmed the importance of multi-path recall and rank fusion. Customer messages, conversation state, preference profiles, and product fields each provide only partial evidence. Processing them as independent retrieval paths and then combining the results is more stable and easier to debug than compressing all information into a single query.

Finally, a high score on a public evaluation set does not guarantee complete generalization in real-world environments. Sound engineering judgment means reporting not only the best results but also the scope of the data, testing boundaries, and known limitations. Further validation using unseen data, more diverse expressions, and system-resource benchmarks is still required before real-world deployment.

7. What’s Next for Shopping Copilot AI

In the next phase, we will continue developing Shopping Copilot AI in three areas: generalization, runtime efficiency, and production readiness.

First, we will build a larger independent test set that is not used for parameter tuning during development. It will include customer expressions that more closely resemble real-world scenarios, such as paraphrases, spelling errors, multilingual input, implicit preferences, and out-of-distribution products. This will help prevent the system from overfitting to the public evaluation data and provide a more accurate assessment of its ability to generalize to new scenarios.

Second, we plan to introduce lightweight local semantic retrieval as an additional recall path for handling out-of-vocabulary terms and variations in phrasing. The existing deterministic constraint system will remain in place as a safety boundary and fallback mechanism. This hybrid architecture can expand language coverage while preserving explainability and output stability.

Third, we will replace the in-memory index with a persistent, prebuilt index and systematically measure startup time, query latency, peak memory usage, and concurrent performance. Fixed ranking weights and confidence thresholds will also be calibrated on an independent development set and then validated against unseen data.

Finally, we hope to expand Shopping Copilot AI from a competition prototype into a shopping assistant that can integrate with real e-commerce platforms. It could connect to real-time inventory, pricing, product variants, and customer feedback while supporting merchant-configurable business rules.

Our long-term goal is to let customers express their needs progressively through natural language and help them find products that genuinely fit—with fewer searches, less manual filtering, and greater confidence.