Adaptive Shopping Copilot

A stateful shopping copilot combining hybrid retrieval, adaptive clarification, intent-aware reranking, and failure-aware semantic exploration for multi-turn product search.

Comment

Inspiration

Online shopping search often assumes that users already know exactly what they want. In reality, shopping conversations are much more dynamic.

A user may begin with a vague request, gradually reveal preferences, reject an attribute, or even replace an earlier requirement entirely. A useful shopping copilot therefore needs more than a single semantic-search query. It needs to maintain conversational state, understand changing intent, decide when to ask clarification questions, and adapt its retrieval strategy when previous recommendations fail.

This motivated Adaptive Shopping Copilot, a stateful conversational retrieval system developed for TikTok TechJam 2026 Track 4: Shopping Copilot - AI Conversational Search and Recommendations.

The main goal was to combine strong retrieval precision with controlled semantic exploration while remaining lightweight, transparent, reproducible, and capable of running fully offline.

What it does

Adaptive Shopping Copilot maintains an isolated state for every shopping session.

It tracks information such as:

  • the current product category;
  • hard constraints;
  • soft preferences;
  • structured attributes such as material, color, size, style, use case, and budget;
  • previously asked clarification questions;
  • attributes the user explicitly says do not matter;
  • preference overrides;
  • a weak long-term profile signal used only for clarification decisions.

The current conversational state is used to dynamically route the session between two behaviors.

In Buying mode, when explicit hard constraints are available, the system becomes more precision-oriented and prioritizes products that satisfy accumulated requirements.

In Browsing mode, when no hard constraint has yet been established, the system retains a stronger lexical anchor while allowing greater semantic exploration.

The agent also supports multi-turn preference accumulation, no-preference responses, intent overrides, adaptive clarification, and recovery when earlier recommendations fail.

How we built it

The final V15 system uses a hybrid conversational retrieval pipeline.

1. BM25 lexical retrieval

SQLite FTS5 provides an in-memory BM25 search index over the frozen 50,000-product catalog.

The indexed product metadata includes:

  • title;
  • categories;
  • features;
  • details;
  • store;
  • description.

Field-specific BM25 weights are used so that more informative metadata contributes more strongly to lexical retrieval.

2. MiniLM semantic retrieval

The system uses sentence-transformers/all-MiniLM-L6-v2 as a local sentence encoder.

The 50,000 catalog products are encoded offline into 384-dimensional embeddings. At runtime, only the current conversational query needs to be encoded, and semantic similarity is calculated locally with NumPy.

3. Hybrid candidate fusion

A route-dependent lexical prefix is preserved as a precision anchor.

The remaining BM25 candidates and MiniLM dense candidates are combined using weighted Reciprocal Rank Fusion (RRF).

The lexical signal remains dominant, while dense retrieval provides additional semantic recall when the user's wording does not exactly match catalog metadata.

4. Constraint-aware reranking

The hybrid candidate pool is reranked using accumulated conversational evidence.

The priority is:

Hard-constraint matches

Soft-preference matches
Original hybrid retrieval rank

This ensures that explicit current-session requirements remain stronger than weaker preference signals.

5. Adaptive clarification

The agent can recommend products and ask one structured clarification question in the same turn.

During the first few turns, it uses a stable clarification order to gather useful constraints quickly.

Later in the conversation, clarification becomes candidate-aware. The system estimates the usefulness of asking about an attribute using:

Question Value = Candidate Coverage × Attribute Diversity

This favors attributes that are both well represented among current candidates and useful for separating them.

A historical user profile is used only as a weak tie-break when two clarification choices have nearly equal information value. Current-session evidence always takes priority.

6. Failure-aware retrieval recovery

One of the key ideas developed during the project was to treat continued interaction as an implicit failure signal.

If a conversation reaches Turn 4, earlier recommendation rounds have not yet located the target in the evaluation setting.

The agent therefore changes its workflow:

  • Turns 1-3: retrieve up to 50 candidates.
  • Turn 4+: expand the candidate pool to 100.

This increases recall only when deeper retrieval is actually needed.

7. Semantic diversity for late browsing sessions

The 50,000 catalog embeddings are clustered offline into 128 semantic groups using MiniBatchKMeans.

For late-stage Browsing sessions:

  • Rank 1 is preserved as a precision anchor.
  • Ranks 2-10 preferentially explore candidates from previously unseen semantic clusters.
  • If there are not enough unique clusters, the remaining positions are filled using the original ranking.

This creates a controlled precision-versus-exploration trade-off only after earlier recommendations have failed.

8. Offline and failure-safe execution

The final system does not depend on a paid LLM API.

The MiniLM model and precomputed embeddings can be loaded locally, and the complete hybrid pipeline was successfully reproduced with network access disabled.

If dense retrieval assets are unavailable, the agent falls back to BM25-only retrieval rather than failing.

Challenges we ran into

One of the biggest challenges was balancing precision and semantic exploration.

Dense-only retrieval performed substantially worse than the hybrid approach. Increasing the semantic contribution too aggressively also reduced ranking quality, which showed that dense retrieval worked better as a complementary recall signal rather than a replacement for strong lexical search.

Another challenge was deciding how much to personalize recommendations. Directly reranking products based on historical user-profile preferences reduced public-development performance because historical preferences could conflict with the user's current request. The final design therefore uses profile information only as a weak clarification tie-break.

Price handling was another useful lesson. I tested explicit product-price filtering and price-aware reranking, but both reduced public-development ranking quality. These mechanisms were removed from the final system. Budget can still be represented as conversational state, but product price is not used as an explicit ranking signal in V15.

Intent override handling was also challenging. The agent needed to remove outdated preferences when users changed their minds rather than simply accumulating contradictory constraints.

Finally, the project had to remain reproducible under possible network restrictions. This required a local MiniLM model, precomputed embeddings, an explicit BM25 fallback, and zero dependence on external inference APIs.

Accomplishments that we're proud of

The final V15 system achieves the following results on the official 200-session public development set:

Metric Result
Hit Rate@10 0.925
MRR 0.606480
MTTC 3.435
Efficiency 0.7565
TechnicalScore 0.795744

Scenario-level Hit Rate@10:

Scenario Hit Rate@10
Buying 0.9375
Browsing 0.9625
Intent Override 0.8000
Boundary 0.9000

For comparison, the provided weak BM25 starter achieved a TechnicalScore of approximately 0.106710 on the same public development set.

I am also proud that the final system:

  • maintains conversational state across multiple turns;
  • handles intent overrides and no-preference responses;
  • adapts its retrieval strategy after repeated failure;
  • combines lexical precision with semantic exploration;
  • runs without a paid LLM API;
  • reports zero prompt and completion tokens;
  • can run fully offline;
  • includes an independently tested BM25 fallback;
  • is reproducible through a public repository and separately distributed runtime assets.

The complete 200-session evaluator ran in approximately 209.7 seconds on an Intel Core Ultra 5 125H machine, including model loading, index construction, agent execution, and evaluator overhead.

These results are public-development measurements only and are not claims about the organizer's private evaluation set.

What we learned

The biggest lesson was that conversational state management can matter more than simply adding a more powerful semantic model.

The transition from a weak baseline to structured multi-turn state tracking and clarification produced a much larger improvement than dense retrieval alone.

I also learned that hybrid retrieval works best when different retrieval signals have clearly defined roles. BM25 provides strong precision, while MiniLM contributes semantic recall.

Another important lesson was that conversation length itself can become runtime context. Reaching later turns can indicate that the current retrieval strategy is not working, which can justify switching to a broader candidate pool and more exploratory recommendation strategy.

The experiments also showed that personalization should be applied conservatively. Historical preferences are useful, but they should not override stronger evidence from the current conversation.

Most importantly, systematic ablation was essential. Several features that sounded useful in theory, including direct profile reranking and price-aware ranking, were removed because measured performance showed that they made the system worse.

What's next for Adaptive Shopping Copilot

With more development time, I would extend Adaptive Shopping Copilot in several directions.

First, I would explore stronger semantic constraint parsing so that the agent can understand more diverse natural-language expressions without relying as heavily on rule-based patterns.

Second, I would investigate uncertainty-aware Buying/Browsing routing rather than using only the presence of hard constraints.

Third, clarification could be improved using a learned policy that estimates expected information gain directly from historical interaction data.

I would also explore richer semantic compatibility scoring between product metadata and hard constraints, while keeping explicit current-session intent dominant.

For a production environment, the embedding and semantic-cluster pipeline would need incremental refresh support so that new catalog products could be added without rebuilding everything from scratch.

Finally, I would evaluate the system on a broader range of conversational language, noisier user input, and more diverse product categories.

The current version focuses on a lightweight and transparent architecture that demonstrates how conversational state, hybrid retrieval, adaptive clarification, and failure-aware semantic exploration can work together in a practical shopping copilot.

Built With

Share this project:

Updates

Submission history