ShopShift: An Adaptive Shopping Copilot

Inspiration

Most e‑commerce search engines treat every message as an isolated query. Real shoppers do not behave that way. They start with vague ideas, discover preferences while browsing, reject earlier assumptions, and sometimes change direction completely. A request such as “I’m just exploring” may become “Actually, it must be waterproof and under $80” one turn later.

A conventional system that keeps appending keywords will carry outdated constraints forward and return increasingly irrelevant results. This inspired us to build ShopShift, a shopping copilot based on a simple principle:

A useful conversational search system must know not only what to remember, but also what to forget.

Instead of forcing every shopper through one fixed pipeline, ShopShift continuously determines whether the user is browsing, buying, or overriding an earlier intent. It then adapts its dialogue and retrieval strategy at runtime.

What ShopShift Does

ShopShift transforms each conversation into an evolving search plan. On every turn, it:

Detects whether the shopper is browsing, buying, or changing direction.
Extracts the product category, hard requirements, soft preferences, budget constraints, and “no preference” answers.
Updates a session state that separates persistent context from intent‑specific information.
Distills the conversation into a compact retrieval context.
Searches the frozen 50,000‑product catalog through multiple specialized routes.
Fuses the candidate lists and reranks them using retrieval, constraint, dialogue, and product features.
Returns the strongest matches or asks one high‑value clarification question when the search space remains too broad.

When the user explicitly changes direction, ShopShift performs selective forgetting. It removes obsolete hard and soft constraints while preserving stable information such as the product category and anonymized profile signals. This prevents the conversation from being “haunted” by requirements the shopper has already abandoned.

ShopShift also treats clarification as part of ranking rather than generic chatbot small talk. It remembers which attributes have already been discussed, understands when the shopper has no preference, and chooses the next unanswered attribute most likely to narrow the candidate pool.

For an ambiguous first request, the system can run retrieval internally but briefly defer an uncertain recommendation list until it receives one useful preference. This avoids locking the correct product into a weak early rank.

How We Built It

We implemented ShopShift as a modular, offline‑first Python system compatible with the competition’s official Agent interface.

1. Intent routing

A deterministic intent router separates conversations into three situations:

Browsing: the user is exploring possibilities.
Buying: the user has concrete requirements or purchase intent.
Intent override: the user explicitly changes or invalidates an earlier request.

Each route receives different retrieval weights and state‑update behavior.

2. Conversational state

A per‑session state machine tracks:

Product category
Hard constraints
Soft preferences
Previously asked attributes
“No preference” responses
Dialogue history
User profile signals
Current candidate‑pool size

Hard constraints dominate the final ranking, while broad profile tags remain weak ranking signals rather than strict filters.

3. Dynamic context distillation

After every turn, ShopShift rebuilds a compact search context from the current message, accumulated slots, user profile, and active intent. This runtime context programming allows the retrieval workflow to evolve with the conversation instead of searching a static concatenation of all previous messages.

4. Multi‑route retrieval

We built an in‑memory catalog index with SQLite FTS5 and weighted BM25 retrieval. Candidates are collected through several routes:

Current‑message keyword retrieval
Product‑category retrieval
Accumulated‑constraint retrieval
Conjunctive retrieval for complete feature fragments
An optional semantic retrieval adapter

The broad routes maximize recall, while the conjunctive route prioritizes products containing complete, discriminative phrases. For example, a rare feature such as “dual gore panels” receives more weight than a common term such as “leather.”

The semantic route is exposed through a small adapter interface. This allows a local dense retriever to be added later without changing the dialogue, state, or reranking components. When no semantic model is installed, the rest of the pipeline remains fully functional.

5. Candidate fusion

The retrieved lists are combined using Reciprocal Rank Fusion (RRF):

$$ \operatorname{RRF}(d)=\sum_{r\in R}\frac{w_r}{k+\operatorname{rank}_r(d)} $$

where R is the set of retrieval routes, w_r is the intent‑dependent weight of route r, and k controls how quickly rank influence decays.

Because broad category retrieval can overwhelm more precise evidence, ShopShift explicitly seeds the strongest keyword and constraint candidates into the reranking pool. This prevents a strong BM25 match from disappearing before the final ranking stage.

6. Learned reranking

Every candidate is represented by a shared 36‑feature vector covering:

Hard‑ and soft‑constraint satisfaction
Exact phrase matches
IDF‑weighted evidence
Category overlap
Rank and score from each retrieval route
RRF score
Cross‑route agreement
Product rating and popularity
Price and budget compatibility
Profile affinity
Dialogue turn
Buying‑versus‑browsing intent

We tested manual scoring, logistic regression, random forest, and LightGBM LambdaRank using the same feature representation. This allowed us to compare ranking methods without quietly changing the information available to each model.

Our production‑oriented default is a compact 11‑feature logistic reranker. Its coefficients and standardization statistics are stored in JSON, so inference requires no hosted model, API key, network connection, or scikit‑learn installation.

We also developed two optional LightGBM LambdaRank models:

A precise model operating on approximately 120 candidates
An efficiency‑oriented wide model operating on approximately 300 candidates

These higher‑capacity models remain explicit experimental options rather than the automatic default.

Results

We evaluated ShopShift using the official deterministic evaluator on all 200 public development sessions.

Metric	Weak starter	ShopShift default
Hit Rate@10	0.1250	0.9950
MRR	0.0680	0.7910
MTTC	9.81	2.51
Efficiency	0.1190	0.8490
TechnicalScore	0.106710	0.904600

The default system finds the purchased product in its Top 10 for 99.5% of public development sessions and reduces mean turns to conversion by approximately 74% compared with the starter agent.

Our experimental wide LambdaRank configuration achieved:

1.0000 Hit Rate@10
0.9975 MRR
1.69 MTTC
0.9310 Efficiency
0.985450 TechnicalScore

However, this was a full‑fit result on the public development set. Profile‑grouped cross‑validation revealed a meaningful generalization gap. We therefore kept the more stable logistic model as the default; its grouped cross‑validation TechnicalScore was:

$$ 0.9054 \pm 0.0159 $$

This decision reflects an important principle behind the project: the highest public score is not automatically the best production choice.

Challenges We Faced

Balancing recall and precision

Broad OR‑based retrieval produced strong coverage but could bury exact products beneath category‑level matches. We addressed this with separate broad and conjunctive routes, IDF‑weighted feature fragments, precise‑route candidate seeding, and learned reranking.

Reducing turns without committing too early

Returning recommendations immediately can reduce apparent latency while locking the correct product into a poor reciprocal rank. Our adaptive deferral policy still performs retrieval on the first turn, but can hold an uncertain list long enough to ask one targeted question.

Handling intent overrides safely

Appending new requirements caused contradictory searches, while resetting the entire session discarded valuable context. We introduced selective forgetting: intent‑specific slots are cleared, but stable category and profile information survives.

Learning from only 200 public sessions

High‑capacity ranking models produced impressive full‑fit scores but could memorize patterns associated with public users and products. We used session‑ and profile‑grouped validation, feature‑selection experiments, coefficient‑stability analysis, and explicit model modes to distinguish leaderboard optimization from likely private‑set performance.

Keeping inference reproducible

The organizer environment cannot depend on paid APIs or hosted infrastructure. We therefore packaged a pure‑Python linear inference path, built the search index entirely in memory, and provided deterministic fallbacks for optional dependencies.

What We Learned

We learned that conversational recommendation quality is not determined by the final reranker alone. A strong ranker cannot recover a product that never reaches the candidate pool. A high‑recall retriever cannot repair a conversation carrying obsolete constraints. One well‑chosen clarification question can improve the final ranking more than another layer of model complexity.

Most importantly, we learned that memory is useful only when paired with deliberate forgetting.

We also learned to treat evaluation design as part of the product. The experimental model with the highest public score was not the model we trusted most. Grouped cross‑validation changed our final architecture and led us toward a simpler, more reproducible default.

What Comes Next

With more time, we would:

Add a small local bi‑encoder and cross‑encoder through the existing semantic adapter
Compare dense and sparse retrieval under identical latency and memory limits
Replace heuristic clarification priorities with expected information gain
Expand slot extraction beyond the competition’s clean‑text assumptions
Calibrate confidence and deferral decisions on additional held‑out users
Provide clearer explanations of why each product was recommended

ShopShift is ultimately more than a catalog ranker. It is a compact decision engine for conversations in which the destination becomes clear only while the shopper is moving.