Shopping Copilot by KI

Inspiration

Shopping rarely begins with a perfect query. A customer may start with a vague need, disclose an important feature only after seeing options, or change direction completely. A stateless ranker can forget useful context; an append-only conversational system can make the opposite mistake and let obsolete preferences contaminate the new request.

We built Shopping Copilot around one idea: ask better questions, preserve active intent, and use every disclosed catalog signal to rank the right product earlier. Clarification turns and Top-10 positions are both scarce resources, so the Agent should spend them deliberately.

What it does

Shopping Copilot is a multi-turn product-search Agent for Track 4. It returns up to ten unique products from the frozen 50,000-item catalog while optionally asking one structured clarification question. Four contributions work together:

Constraint Ledger. Requirements are stored with their slot, source turn, status, and provenance. Active, superseded, negative, declined, and exhausted state remain distinct. A true “no preference” can retract a slot, while “no additional preference” stops further questioning without erasing evidence already supplied.
Question Value Estimator. For each legal unasked attribute, the Agent combines candidate metadata coverage, balanced partition value, route relevance, candidate-score uncertainty, prior questions, declines, and remaining turns. It asks only when the bounded estimate is useful, and it can still recommend products in the same turn.
Catalog Evidence Retrieval. A read-only Evidence Fingerprint index matches complete catalog-visible feature strings, detail values, material, color, and available price evidence. This gives a high-confidence local route for exact disclosed evidence without claiming neural semantic understanding.
Adaptive Candidate Portfolio. High ranks form a precision core. When intent is uncertain, only lower positions form a bounded exploration tail covering distinct plausible facets. As constraints become clearer, the portfolio shifts back toward precision.

Buying, Browsing, and Intent Override routes share the same stateful pipeline. Replacements remove stale category-dependent evidence, current intent receives priority, and every returned parent_asin is validated against the frozen catalog.

How we built it

The official entry point is the required Python interface: from starter.agent import Agent, followed by reset() and respond(). The core uses Python’s standard library and an in-memory SQLite FTS5 index. Weighted Reciprocal Rank Fusion combines bounded routes for the current message, active constraints, stable context, safe anonymous profile hints, and exact catalog evidence. A deterministic constraint-aware reranker, Question Value Estimator, and Adaptive Candidate Portfolio produce the final response.

The selected policy is protocol_evidence; the earlier deterministic phase3 policy remains a clean fallback. The official evaluator imports only the Python Agent. FastAPI is an optional thin adapter, and the React, TypeScript, Vite, and Ant Design interface is a demonstration layer over the same core—not part of scoring.

We began by reproducing the official starter baseline, then used component ablations, generic contract tests, and catalog-derived fixed-seed diagnostics to decide what to keep. We preserved official files byte-for-byte, kept the catalog immutable, and separated aggregate public evidence from private development outputs.

Challenges we ran into

The hardest dialogue problem was replacement scope: when a customer switches from a backpack to walking shoes, the Agent must remove obsolete category evidence while retaining compatible information such as a budget. Another challenge was distinguishing declined attributes from exhausted questioning without seeing evaluator labels at runtime.

Ranking created a different trade-off. Many misses were retrieved but remained below the Top 10, so expanding recall alone was not enough. We needed to improve lower-rank coverage without sacrificing the MRR value of the highest positions.

We also tested a guarded DeepSeek candidate reranker during development. It could only reorder a bounded candidate set, never generate an ASIN, and always fell back to the selected offline ranking on failure. The experiment cost approximately US$2.08. A 200-session catalog-derived screening improved TechnicalScore from 0.836591 to 0.841916, but that result was not the official public set. On a separate 800-session exact-policy unseen-target confirmation—not the organizer final set—the gain shrank to 0.000993, MRR fell from 0.683708 to 0.682518, Buying lost two hits, and strict JSON validity was only 844/862 (97.91%). We rejected the added model, network, credential, and reliability complexity rather than keeping it for a negligible larger-set gain.

Accomplishments that we're proud of

The unchanged official 200-session public evaluator produced:

Metric	Official starter baseline	Selected runtime
Hits	25 / 200	188 / 200
Hit Rate@10	0.125000	0.940000
MRR	0.068034	0.645704
MTTC (lower is better)	9.810000	3.985000
Recommended TechnicalScore	0.106710	0.804011

The selected Agent reached 76/80 Buying, 77/80 Browsing, 25/30 Intent Override, and 10/10 Boundary hits. It passed 78 selected-runtime hardening tests. The canonical public-evaluator run used zero model tokens, made no runtime network calls, incurred US$0 external-service cost, and completed in 70.288 seconds with approximately 642.070 MiB peak process-tree working set. A standalone fresh clone reproduced every aggregate metric exactly.

What we learned

Most late-stage failures were not pure recall failures. The target often entered a route but ranked below the cutoff. That evidence moved our work from generic candidate expansion toward dialogue state, constraint provenance, candidate-aware questions, and rank-protected set construction.

To strengthen our conceptual understanding, we reviewed selected Stanford CS276 material and sections of Introduction to Information Retrieval, consulted Jurafsky and Martin’s frame-based dialogue systems chapter, the original Reciprocal Rank Fusion paper, Berkeley CS189 material on entropy and information gain, and SQLite’s FTS5 documentation. The references connected to the project in specific ways:

CS276 and IIR clarified inverted indexes, candidate retrieval, ranking, and evaluation;
frame-based dialogue provided the vocabulary for slots and conversation state;
the RRF paper grounded multi-route rank fusion, while our route weights and rank constant remained project-specific choices;
entropy and information-gain material helped us reason about Question Value, while the project’s partition_gain remains a bounded balance heuristic rather than standard supervised information gain;
SQLite’s FTS5 documentation defined the actual sparse-index, tokenizer, query, and BM25 behavior used by the runtime.

The DeepSeek LLM experiment reinforced a broader lesson: permission to use an API is not evidence that an API improves the submitted system. A small screening gain must survive larger generalization, scenario, reliability, and cost gates. For this competition evidence, it did not.

What's next for Shopping Copilot by KI

For a production environment with noisier free-form, multilingual, or long-tail queries, a model could still be useful for isolated intent normalization or query rewriting. That is production speculation, not a final result. Any future experiment would keep catalog-bounded identifiers, deterministic fallback behavior, explicit cost and reliability gates, and a clear comparison against the current offline control. We would also explore reproducible persisted indexes, lower cold-start memory, broader accessibility testing, and clearer user-facing evidence explanations.