End-to-end architecture of 0x00 Shopping Copilot: State → Retrieval → Evidence-Aware Reranking → Dialogue → Recommendations.
Frontend demo interface visualizing the shopping session, agent state, recommendations, and pipeline behavior in real time.
A complete multi-turn shopping conversation showing clarification, evolving constraints, and progressively refined recommendations.
Public-200 evaluation results: 0.995 HitRate@10, 0.9608 MRR, 2.23 MTTC, and a 0.96115 Technical Score.

0x00 Shopping Copilot

Inspiration

A shopper can start with “I need a pair of shoes,” mention a budget later, and change their mind after seeing a few options. Each message changes what a useful recommendation looks like.

That made us think about shopping search differently. A conversational assistant needs more than a good answer to the latest query: it needs a working memory of the customer's requirements, a way to revise that memory, and a reason for choosing its next question.

We built 0x00 Shopping Copilot for TikTok TechJam 2026 Track 4 around one idea:

Conversation is an evolving source of search evidence, not just a chat interface around a product index.

The assistant keeps track of what the shopper wants, updates its recommendations as those needs change, and asks follow-up questions to narrow the search. Its core search and ranking run locally. Optional model APIs help interpret ambiguous requests and add a semantic search step later in the conversation.

What it does

Shopping Copilot searches the competition's fixed catalog of 50,000 products over conversations of at most 10 turns. On each turn, it updates the shopper's requirements, finds candidate products, and ranks them using information in the catalog. It can recommend products and ask a follow-up question in the same response.

The benchmark contains four types of simulated conversations:

Buying: the shopper gives specific requirements early.
Browsing: the shopper starts with a broad request.
Intent Override: the shopper changes an earlier preference during the conversation.
Boundary: the shopper may explicitly say they have no preference for an attribute.

Our Demo Mode lets viewers follow a complete conversation generated by the official simulator, see the recommended catalog products, and inspect how the agent's understanding changes. Recommendations use the official parent_asin product identifiers; the system does not make purchases.

The benchmark generates conversations from catalog metadata. The agent receives the published session inputs and a summary of the shopper's profile. It is not given the target product's identity, the scenario label, or the evaluator's hit status.

How we built it

We split the agent into four parts: remembering the shopper's requirements, finding products, ranking them, and choosing what to ask next.

1. Explicit shopping state

Each conversation has a ShoppingState. It records whether the shopper is buying or browsing, what they require or prefer, what they have rejected, and which attributes they do not care about. It also keeps the conversation history and a record of questions already asked.

We track where each requirement came from so the agent can distinguish an earlier preference from its replacement. When the shopper changes direction, the agent starts a new constraint epoch, or version of their requirements. Retrieval and ranking can then stop relying on stale soft preferences, and the agent resets its memory of recommended products. This is not a complete reset of the shopper's needs: explicitly replaced constraints are archived, while other hard requirements can carry forward.

When the agent recognizes that a shopper has no preference for an attribute, it removes that constraint and stops selecting that attribute for follow-up questions. Having no preference is different from rejecting a particular value, so we keep the two separate.

Rules handle the main state updates. When enabled, Alibaba Cloud DashScope Qwen Plus can help interpret ambiguous language. We check the structure and confidence of its responses before using them, and rule-derived values take priority when both methods update the same constraint field. The optional resolver also includes error handling so it can keep the rule-based update when semantic resolution fails.

2. Finding products with BM25 and optional semantic search

We use in-memory SQLite FTS5 indexes to retrieve products with BM25. Title and category matches receive strong weight; features, details, store names, and descriptions provide more information to search against.

On the first browsing turn, the search uses only a concise title-and-category representation. Buying requests and later browsing turns use the more detailed product text. Depending on the stage of the conversation, we select a window of results and pass up to 100 candidates to the reranker.

Semantic search provides a small, optional supplement during a scheduled later exploration stage. BM25 remains the main retrieval method: every semantic addition must also appear in the broader BM25 results. We call this a lexical-gated residual, since embeddings can suggest additional candidates but cannot bypass the requirement for lexical evidence.

This option uses Alibaba Cloud text-embedding-v4 and precomputed, normalized 256-dimensional product vectors. Before using the cache, we check its model, text representation, catalog fingerprint, and ASIN ordering. If the required configuration is missing or the cache is incompatible, the agent continues with BM25. When the query error handler catches a provider failure, it returns the BM25 results as well.

The evaluation with APIs enabled, reported below, used both optional components: Qwen to interpret state updates and a query-embedding call for semantic retrieval. BM25 retrieval, evidence-coverage reranking, and clarification still ran locally.

3. Evidence-coverage reranking

Once retrieval has found plausible products, our local EvidenceCoverageReranker asks a simple question: how well does each product match what the shopper has told us?

To score each product, we combine:

coverage of the shopper's active hard and soft requirements;
IDF calculated over the current candidates, giving more weight to less common terms;
matches to separate parts of a requirement, so matching one part does not earn full credit for the whole request;
phrase matches, exact values from catalog fields, and matches in the title and category;
whether the price meets a budget or is close to an approximate target, when the request clearly refers to money.

Hard requirements generally count more than soft preferences, with category handled separately. We do not penalize a product as over budget simply because its price is missing. Retrieval chooses which products reach this stage; its score also serves as a late tie-breaker after the main evidence signals.

We explored explicit constraint-matching features in a diagnostic baseline, then chose weighted evidence coverage for the production reranker. The production version does not yet apply explicit rejected-value penalties; those penalties and stronger contradiction checks are still on our roadmap.

4. Broad-to-Specific Dialogue Strategy

Shopping Copilot begins with bounded open discovery: a three-turn sequence of progressively worded questions that gives shoppers room to describe their needs before the agent asks about individual attributes.

Must-have requirements: “Are there any must-have details I should prioritize?”
Broader preferences: “What features or preferences matter most to you?”
Missing requirements and deal-breakers: “Any final requirements or deal-breakers I should consider?”

The aim is to avoid narrowing the conversation too early. A shopper's answer can span category, material, budget, style, and use case, or correct an earlier assumption. The three-turn boundary gives this open discovery phase a clear endpoint.

All three questions use ask_attribute="other", the interface label for an open-ended question that is not restricted to one named attribute. The wording is progressive, while the structural label remains other. Each user-facing question invites a different kind of detail.

After this discovery phase, the agent switches to targeted, candidate-informed clarification. It selects an unresolved attribute using a heuristic that considers how many candidates have that attribute and how much their values vary, with more weight given to higher-ranked products. Attribute priorities, profile preference tags, and what has already been asked or resolved also inform the choice. Questions can include example values from the candidates, helping the shopper compare concrete options. The final response contains recommendations without another question.

We start with the highest-ranked unseen product and move to a broader shortlist in later stages. The agent remembers what it has shown within the current epoch and avoids repeating those products. Later exploration also brings deeper-ranked candidates into consideration, extending the search beyond the leading results.

Detailed question settings, recommendation counts, and retrieval windows are documented in our technical report.

Engineering and development workflow

We built the agent with Python, SQLite FTS5, NumPy, the OpenAI-compatible SDK, and python-dotenv, with DashScope services as an option. The frontend uses FastAPI, Uvicorn, and vanilla HTML/CSS/JavaScript, and we distribute the optional vector matrix through Git LFS.

Shared Item, Candidate, and RankedCandidate classes let the modules exchange products without redefining their structure at each step. Demo Mode and the official evaluator both use the production starter.Agent. A separate Developer Mode lets us inspect a diagnostic baseline step by step, so its ranking scores should not be read as the production reranker's internal scores.

During development, we used replay diagnostics, experiment records, and regression tests to compare changes and inspect what happened between stages. Our final submission keeps the core agent, frontend, and official participant toolkit. We validate it with the unmodified organizer-provided evaluator.

The frozen clothing, shoes, and jewelry catalog comes from Amazon Reviews 2023, published by McAuley Lab at UCSD. The profiles supplied to participants are aggregate summaries, not raw purchase histories or review text. See the dataset attribution.

Challenges we ran into

Deciding what should remain in memory

Accumulating every message is easy; deciding which information is still relevant is harder. A changed preference should not compete equally with its replacement, but a new product category does not necessarily invalidate the budget. Tracking where each requirement came from made it easier to see what the agent was still using. Deciding what to keep when different requirements conflict remains a challenge.

Separating matching words from matching requirements

A product may contain the right words without meeting the shopper's needs. We broke requirements into smaller parts, checked phrases as well as individual words, and compared prices with the stated budget. These checks help us look beyond word overlap, but incomplete descriptions and ambiguous wording still make some products difficult to judge.

One tricky case was telling dimensions apart from prices. “Up to 30 mm wide” describes a size limit, not a budget. Our reranker checks that a request refers to money before using an extracted budget. This guards against a mistaken price penalty, but it does not fix the underlying state: the parser still needs to distinguish units more reliably.

Making external models optional

External models need API keys, add cost and latency, and can fail. We made both model-assisted state updates and semantic search optional, and documented how the system falls back to local methods. This also lets us compare runs with and without APIs, rather than treating model access as a requirement.

Avoiding overfitting to familiar conversations

The public conversations helped us understand the trade-off between early precision, recommendation breadth, clarification, and deeper exploration. We also had to be careful about tuning around familiar templates. A change that helps on the public set may not help when a shopper phrases the same request differently.

Accomplishments that we're proud of

With the optional APIs enabled, our agent found the target product in 199 of 200 public development sessions. We're also proud of bringing the modules together into a working demo, with documented setups for running the system with or without external services.

We ran this evaluation on August 31, 2026, using the unmodified official evaluator. The full setup is documented in our technical report. The overall scores matched our earlier run without APIs:

Metric	With APIs enabled	Earlier run without APIs
HitRate@10	0.995000	0.995000
MRR	0.960833	0.960833
MTTC	2.230000	2.230000
TechnicalScore	0.961150	0.961150

A hit means the target appeared in the recommendations within ten turns. MRR reflects how highly it ranked, while MTTC measures how many turns it took to find it. Misses count as zero for MRR and 11 turns for MTTC.

With APIs enabled, the hit rates by scenario were:

Scenario	Sessions	HitRate@10
Buying	80	98.75%
Browsing	80	100%
Intent Override	30	100%
Boundary	10	100%

For context, the organizer's published weak-BM25 baseline reports a TechnicalScore of 0.10671 on the same public set. TechnicalScore combines hit rate, reciprocal rank, and turn efficiency; it is not the overall hackathon judging score.

Across all 200 sessions, the agent made just three model calls:

Component	Calls	Input tokens	Output tokens
Qwen Plus state interpretation	2	2,006	271
`text-embedding-v4` query embedding	1	93	0
Total	3	2,099	271

The official evaluator recorded 2,370 tokens in total, based on usage counts returned by the model providers. BM25 retrieval, reranking, and clarification ran locally without model calls. This total covers the evaluation only; it excludes the earlier work of generating the product-vector cache.

Both optional components were used, but the overall scores matched our earlier run without any model calls. We did not observe an improvement in the aggregate metrics on this public set.

These results come from the public development set, not the final held-out test. The organizer's additional 800 evaluation sessions are released after the submission deadline under the official final-evaluation process. Performance on that set, or with real shoppers, is still an open question.

What we learned

We learned to choose tools for specific jobs. Rules make state updates easier to inspect, and BM25 works well when a request uses words found in the catalog. Embeddings and language models offer ways to handle less direct wording. Keeping the shopping state explicit helps us see how these pieces interact and track down mistakes.

We also learned that clarification is part of search. Each answer gives the next search more information to work with. Choosing what to ask and how many products to show can change the course of the conversation, so we tested those choices alongside the ranking logic.

Recording the code version and settings behind each experiment helped us compare changes fairly and reproduce our results. Those records also helped us explain what the system actually does and where it still falls short.

What's next for 0x00 Shopping Copilot

Our next priorities are to:

explore how candidate uncertainty and search progress could guide the timing of follow-up questions and broader search;
make the state parser better at distinguishing prices from other measurements and resolving conflicting requirements;
add explicit penalties for rejected values to the production reranker;
test paraphrases, spelling errors, short answers, and complex overrides beyond the benchmark templates;
compare the optional models on more varied requests, measuring their effect on accuracy, response time, and cost, including the cost of building the vector cache.

Longer term, we want to search larger catalogs and handle a wider range of conversations while keeping the modules easy to test and replace.

Shopping Copilot is a step toward search that remembers, revises, and asks purposeful questions. Its central lesson is simple: better recommendations can come from managing the conversation's evidence well, not only from choosing a larger model.