Shopping Agent

AI shopping copilot using intent routing, conversational memory, hybrid retrieval and semantic reranking to find the right product faster with fewer turns.

Comment

Inspiration Online shopping rarely begins with a perfect keyword query. People say things like “I need something for a beach birthday,” gradually add preferences, or completely change their minds. Traditional search engines often forget this conversational context or rely too heavily on exact words. We built Shopping Copilot to make product search behave more like a helpful shopping assistant: remember what matters, recognize when preferences change, ask one useful question, and find the right product quickly. What it does Shopping Copilot is an offline conversational product-retrieval agent for a frozen catalog of 50,000 Amazon products. On every turn, it:

  1. Updates the shopper’s session state.
  2. Accumulates new preferences.
  3. Removes preferences that have been overridden.
  4. Identifies Buying, Browsing, or Intent Override behavior.
  5. Asks for clarification when information is insufficient.
  6. Searches the catalog through multiple retrieval routes.
  7. Reranks the candidates and returns the top 10 products. The goal is: [ \text{Right Product} + \text{High Rank} + \text{Few Turns} ]The competition score combines coverage, precision, and efficiency: [ \text{TechnicalScore} =0.50(\text{HitRate@10}) +0.30(\text{MRR}) +0.20(\text{Efficiency}) ]How we built it We followed an evaluation-driven process instead of immediately adding a large language model. Conversational state Each session maintains:
  8. Product category
  9. Active preferences
  10. Buying, Browsing, or Override mode
  11. Previously overridden preferences
  12. Clarification history
  13. An anonymized preference profile When a shopper says, “Actually, ignore my earlier preference,” the obsolete preference is removed while stable information such as the product category is retained. Proactive clarification The agent can ask a question and return recommendations during the same turn. This avoids wasting a conversion opportunity. For broad requests, it asks what other requirements or features matter most. The newly disclosed information is accumulated and used in later retrieval. Multi-route retrieval The catalog is indexed entirely in memory using SQLite FTS5. We use three retrieval routes:
  14. Balanced category-and-preference retrieval
  15. Preference-heavy retrieval
  16. Category-heavy retrieval Each route retrieves up to 300 candidates. Their results are fused before final ranking. Constraint-aware reranking Candidates are reranked using:
  17. Category agreement
  18. Exact preference-phrase matches
  19. Token coverage
  20. Evidence across retrieval routes
  21. A Buying/Override BM25 guardrail
  22. A log-scaled purchase-popularity prior The popularity feature is a tie-breaker, not a replacement for relevance. We tested several weights and selected the value that produced the best complete TechnicalScore rather than maximizing Hit Rate alone. Offline-first design The final system uses:
  23. Python
  24. SQLite FTS5
  25. Python standard library
  26. The official Amazon Reviews 2023-derived competition dataset It requires:
  27. No external API
  28. No API key
  29. No network access during inference
  30. No paid model
  31. Zero prompt or completion tokens Results Using the unchanged official evaluator on the 200 public development sessions: Version Hit@10 MRR MTTC TechnicalScore Organizer BM25 baseline 0.125 0.068034 9.81 0.106710 Shopping Copilot V2.3 0.990 0.759317 1.82 0.906395

These are public development results and do not guarantee identical performance on the hidden evaluation set. Challenges we faced Understanding the evaluator The most important early challenge was understanding exactly how conversations, turns, clarifications, overrides, and ranking metrics were evaluated. Inspecting the official code prevented us from optimizing for subjective chatbot quality instead of the actual task. Preserving evolving intent Blindly concatenating every message caused stale preferences to remain after a shopper changed direction. We introduced explicit override erasure while preserving stable context. Balancing coverage and precision A stronger popularity prior improved coverage to 99.5%, but reduced MRR enough to lower the overall TechnicalScore. This taught us not to optimize one headline metric in isolation. Generic product constraints Many products share descriptions such as “cotton,” “imported,” or “button closure.” We needed multiple retrieval routes and conservative tie-breaking to distinguish highly similar candidates. Rejecting unnecessary complexity We tested structured material/color weights, profile-based ranking, average-rating priors, and several popularity weights. Some produced no improvement or caused regressions, so they were not promoted. What we learned The project showed us that strong conversational search does not automatically require an expensive LLM. The largest improvement came from:

  • Correct state management
  • Asking useful questions
  • Accumulating information
  • Handling overrides
  • Separating retrieval from ranking
  • Measuring every change We also learned that a failed experiment is valuable when it prevents unnecessary complexity. Every accepted component has a measurable reason to exist. What’s next With more time, we would:
  • Test a lightweight dense-retrieval route for semantic scenarios
  • Improve robustness to unseen paraphrases
  • Add richer hard-versus-soft constraint handling
  • Evaluate fairness and cold-start effects from popularity ranking
  • Measure latency and memory across different hardware
  • Validate generalization on a larger held-out set Shopping Copilot demonstrates that an efficient, explainable, offline system can turn evolving conversations into accurate product recommendations—without relying on expensive infrastructure or external model APIs.

Built With

Share this project:

Updates