JamZ Conversational Shopping Agent

Inspiration

Traditional e-commerce search works well when shoppers already know exactly what they want, but real shopping conversations are rarely that simple. Users often begin with vague requirements, reveal preferences gradually, or even change their minds halfway through a conversation.

For TikTok TechJam 2026's Shopping Copilot — AI Conversational Search and Recommendations challenge, we wanted to explore how a shopping agent could treat search as an evolving conversation rather than a sequence of isolated queries.

Our goal was to build an agent that could progressively narrow down a catalog of 50,000 products, ask useful questions, retain relevant information across turns, handle changing intent, and identify the intended product within the challenge's 10-turn limit — while remaining completely offline.

What it does

JamZ is a fully offline conversational shopping agent that helps users find the product they are looking for through an iterative conversation.

Instead of relying on a single search query, JamZ maintains the constraints revealed throughout the conversation and continuously updates its understanding of the shopper's intent.

The agent:

Tracks user preferences and constraints across multiple turns
Detects when a user's intent changes
Searches across a 50,000-product catalog
Uses BM25-based retrieval to generate candidate products
Re-ranks candidates using multiple signals such as exact phrase evidence, category coverage, price, and ratings
Asks for additional information each turn to progressively narrow the search
Controls when and how many recommendations are shown to maximize the likelihood of identifying the correct product quickly

The system operates entirely locally with no LLM API, network access, API keys, or external services.

On the provided 200-session public evaluation set, our agent improved the TechnicalScore from 0.107 for the starter agent to 0.955, achieving a 0.995 HitRate@10, 0.983 MRR, and 2.89 MTTC.

How we built it

We approached the problem as a combination of dialogue-state tracking, information retrieval, re-ranking, and recommendation policy.

At each customer turn, JamZ first updates its dialogue state using the constraints disclosed so far. This allows the system to preserve relevant preferences across turns while also detecting intent overrides when the shopper changes direction.

For retrieval, we build a local SQLite FTS5 index over the 50,000-product catalog. Product information such as titles, categories, features, details, store information, and descriptions is indexed and retrieved using BM25.

The initial retrieval produces a few hundred candidate products. We then apply an additional re-ranking stage using signals including:

IDF-weighted phrase matches
Exact constraint evidence
Category coverage
Price information
Rating priors

One of the most important parts of our approach was the release policy.

Rather than returning a large list of products every turn, JamZ normally recommends one previously unseen product at a time. Because the evaluator stops once the correct target appears and MRR depends on the target's position within the recommendation list, this allows a strong candidate to appear at rank one rather than being buried inside a larger recommendation set. On the final turn, the agent widens the recommendation list as a fallback.

We also built tooling for synthetic datasets, randomized holdouts, cross-validation, demo sessions, score comparisons, and latency profiling so that changes could be measured rather than evaluated only through individual examples.

Tools and resources

Development tools: VS Code, Git, GitHub
Language: Python 3.10+
Database/Search: SQLite FTS5 with BM25
Libraries/frameworks: Python standard library only; no third-party dependencies
APIs: None — JamZ runs fully offline and does not use external APIs or LLM services
Dataset/assets: The 50,000-product catalog provided by the TechJam organizers

Challenges we ran into

One of our biggest challenges was understanding that improving a conversational agent for this benchmark was not simply a matter of improving retrieval accuracy.

The interaction between the customer simulator and the scoring system mattered significantly. For example, the customer reveals additional constraints in response to ask_attribute, so failing to ask a question can prevent the agent from receiving useful information in later turns.

We also discovered that returning more recommendations is not necessarily better. A correct product placed lower in a list produces a worse reciprocal rank than the same product returned alone at rank one. This led us to rethink the recommendation policy separately from the retrieval algorithm.

Another challenge was generalization. The public evaluation set contains only 200 sessions, making it easy to unintentionally optimize too heavily for those examples. To better understand this risk, we created randomized holdouts, synthetic session sets with unseen targets, and cross-validation experiments.

Not every idea improved performance. We experimented with a late-turn escalation strategy that relaxed assumptions and widened retrieval for unresolved sessions. After measuring it, we found that it actually reduced the overall score. Instead of removing the experiment entirely, we kept it disabled in the code and documented the result so that our development process remained reproducible.

We also had to work within the limitations of a fully offline system. Without semantic embeddings or an LLM, matching paraphrases such as "something warm for hiking" with a product described as an "insulated mid-layer" remains difficult when there is little lexical overlap.

Accomplishments that we're proud of

We are particularly proud of improving the official public-set TechnicalScore from 0.107 to 0.955 while keeping the system completely offline and using only the Python standard library.

Our final public evaluation results were:

HitRate@10: 0.995
MRR: 0.983
MTTC: 2.89
TechnicalScore: 0.955

Beyond the score itself, we are proud that the system is reproducible and measurable. We built dedicated tools for testing individual conversational scenarios, comparing the starter agent against our implementation, generating synthetic evaluation sessions, running cross-validation, and profiling latency.

The entire agent also runs without API credentials, network access, or model inference costs, with reported model token usage and model cost both remaining at zero.

What we learned

One of our biggest lessons was that conversational search is not only a retrieval problem.

The quality of the final system depends on several components working together:

What information should we remember? What should we ask next? How should new information modify previous constraints? Which products should we retrieve? How should they be ranked? And when should we actually show them to the user?

We also learned the importance of evaluating system-level behavior rather than assuming that a seemingly more sophisticated approach will perform better.

Several of our improvements came from understanding the interaction between retrieval, conversation state, and the evaluator rather than simply adding more complex models.

At the same time, our experiments highlighted the distinction between benchmark performance and real-world conversational quality. Some strategies perform extremely well against the provided simulator but would not necessarily represent the ideal experience for a real shopper. Recognizing and documenting those limitations became an important part of the project.

What's next for JamZ Conversational Shopping Agent

The biggest next step would be improving JamZ's ability to understand semantically similar requests that use different vocabulary.

We would first introduce dense semantic retrieval alongside BM25, allowing lexical and semantic search results to be combined while maintaining an offline architecture.

We would also replace our current rule-based constraint extraction with a small local model capable of producing structured shopping preferences such as category, color, style, and overridden constraints.

Another improvement would be information-gain-based question selection. Instead of asking the same generic question every turn, the agent could examine its current candidate set and choose the question that is expected to eliminate the greatest uncertainty.

Finally, we would experiment with a cross-encoder re-ranking stage over the strongest retrieved candidates to improve semantic ranking.

Together, these improvements would move JamZ beyond optimization for the competition simulator and toward a more natural, robust conversational shopping system for real users.