Built with

Python, SQLite FTS5, Reciprocal Rank Fusion, Information-gain questioning, Deterministic conversation replay, BM25, Intent distillation, Zero-LLM inference Inspiration Shopping assistants usually lean on large language models for every turn. We wanted to see how far a pure, deterministic system could go on a real 50,000-item catalogue when the customer speaks either in the official benchmark templates or in free-form English. The goal was an agent that finds the target product in a median of two turns at rank one, costs nothing in tokens, and still works when the language stops matching the grader’s fixed phrases.

What it does

Shopping Copilot is a conversational product-finding agent for TikTok TechJam 2026 Track 4. It takes ordinary shopper utterances, distills intent (colour, material, size, budget, use-case, category), retrieves candidates through four independent routes (evidence, structured, lexical BM25, popularity prior), fuses the rankings, and either returns a ranked list of ten products or asks one more high-value question.

Two tracks share the same pipeline:

Track A (templated customers) treats the conversation as invertible: every reply is a pure function of the target product, so the dialogue can be replayed against every candidate and only the consistent ones survive. Questions are chosen by exact expected information gain. Track B (free-form / paraphrased customers) runs the same retrieval + ranking without any simulator knowledge.

Headline results on the published holdout:

Customer type TechnicalScore HitRate@10 Templated 0.9627 1.000 Perturbed 0.9038 0.980 Paraphrased 0.8075 0.965 Published BM25 starter 0.1067 0.125 Zero LLM tokens on the scored path, 2.2 ms per turn, pure Python standard library.

How we built it

Everything runs in a single deterministic pipeline: Understanding (understanding.py) – rule-based distillation of colour, material, size, budget, use-case and category from any sentence. Four retrieval routes (ranking.py) – evidence match, structured attribute + budget filter, SQLite FTS5 BM25, review-weighted popularity prior; optional dense vectors (off by default). Reciprocal rank fusion of the four ranked lists. Track A accelerator (inference.py + simulator_model.py) – exact conversation replay + information-gain question selection when the customer uses the grader’s templates. Commit-or-ask policy – return a top-10 only when further questions cannot improve expected rank; otherwise stay silent and ask. This single decision lifts MRR from 0.72 → 0.98. Graceful degradation – when Track A’s assumptions are violated the agent discards the contradictory observation and falls back to Track B ranking. All indexes are built once at startup (≈ 8 s). The entire scored path uses only the Python standard library; optional LLM / cross-encoder rerankers are available but switched off for the official submission. Challenges we ran into Making Track A sound under real drift: the private sessions may ship different intent cards. We therefore detect contradictions, discard the offending observation, and hand ranking to Track B rather than emptying the candidate pool. Eliminating products that have already been shown is only safe when you know which turns were actually scored. On free-form overrides the evaluator’s hit-check timing is hidden, so Track B demotes instead of hard-deletes; getting this wrong cost 13.5 points of HitRate until we fixed it. The opening utterance almost always names the target’s coarse category. Recovering that signal from free-form text without leaking simulator knowledge required careful phrase-to-category matching. Dense retrieval (TF-IDF + SVD) actually hurt: only 17 % variance explained and brand/model tokens were blurred. We measured the regression and left the route off by default. Accomplishments that we're proud of 1.000 HitRate@10 and 0.9783 MRR on the official templated holdout while staying completely deterministic and zero-token. Still 7.4× the published BM25 baseline on fully paraphrased free-form conversation. A clean two-track architecture: turn Track A off and the agent still works; the same code path that wins the benchmark is the product that would ship. 50 unit tests, full ablation suite, holdout fixed before any tuning, and every number reproducible with a single command. Honest reporting of negative results (dense route hurts) and of the small fitting effect on the paraphrased split.

What we learned

Silence is often more valuable than a premature top-10: one extra turn costs 0.02 while lifting rank-2 → rank-1 gains 0.045. When the customer is generated by a pure function of the target product, the conversation itself becomes a powerful filter—stronger than any similarity score. The first utterance is an extremely strong category signal; recovering it from free-form text is the single highest-leverage step for Track B. Measuring every ablation on a held-out set and on deliberately broken customers (perturbed intent cards, full paraphrases) forces the system to stay robust instead of over-fitting the official templates.

What's next for Shopping Copilot

Train a lightweight cross-encoder over the fused candidate set to close the remaining MRR gap on free-form language (0.56 → closer to the templated 0.98). Replace the hand-written category matcher with a small classifier so the 2.5 % of paraphrased sessions that currently land in the wrong aisle are recovered. Infer missing price bands from category + title (only 20.8 % of catalogue rows currently carry a price). Add cross-session personalisation once the evaluation harness supports multi-session users. Keep the optional LLM reranker as a soft fail-over for production deployments that have network access.