CannotTok

A constraint-aware conversational search agent for TechJam 2026 Problem Statement 4. The agent turns vague or changing shopping requests into ranked products from a frozen 50,000-item Amazon catalog.

Comment

Team CannotTok

Shopping Copilot: Constraint-Aware Conversational Product Search

One-line description

A local shopping agent that remembers constraints, handles changing intent, and finds the target product in 2.98 turns on average without API calls or model inference.

Project links

Inspiration

Shopping rarely starts with a perfect query. A customer might begin with "something for winter," reveal a material preference later, and then change their mind about style. Standard search treats those messages as separate keyword boxes. It forgets what matters and keeps showing near-matches.

We built Shopping Copilot to treat the conversation as the search interface. It keeps a compact state of disclosed constraints, recognizes when an earlier preference has been replaced, and uses each new answer to narrow a fixed product catalog.

What it does

Shopping Copilot works against the frozen 50,000-product Amazon catalog supplied for TechJam. On every turn it:

  1. updates the active category and constraints;
  2. preserves unfamiliar or paraphrased text as bounded fallback context;
  3. asks an open clarification question so useful information arrives quickly;
  4. retrieves 200 candidates using an in-memory BM25 index;
  5. reranks candidates by exact disclosed-constraint coverage and budget fit; and
  6. returns up to ten valid parent_asin values through the official Agent contract.

Intent Override sessions receive special handling. When a customer explicitly changes direction, the stale turn-one preference is removed before the next search. Previously disclosed hard constraints remain intact.

How we built it

The team first implemented a broader research stack with Buying and Browsing routes, context distillation, local embeddings, vector retrieval, reciprocal-rank fusion, and neural reranking.

We then treated the official evaluator as an engineering decision tool rather than a final checkbox. Eight controlled phases isolated the effect of conversation state, enriched index text, vector retrieval, phrase matching, budget checks, reranking, and candidate depth.

The best submission policy was simpler than the largest architecture:

conversation state
  -> in-memory BM25 top 200
  -> deterministic phrase and budget reranking
  -> top 10 products

The selected path is fully local. It requires no credentials, network access, vector artifact, API tokens, or model inference.

Results

On the unchanged 200-session public evaluator:

  • HitRate@10: 0.925
  • MRR: 0.656911
  • Mean Turns to Conversion: 2.98
  • Efficiency: 0.802
  • Technical Score: 0.819973
  • Token use and marginal API cost: 0

Scenario HitRate@10 was 1.00 for Boundary, 0.9375 for Browsing, 0.9125 for Buying, and 0.90 for Intent Override.

The organizer's weak stateless BM25 baseline scored 0.10671. Our final score was 0.819973. The largest gains came from remembering the conversation, asking for high-value information, and checking disclosed constraints directly.

What makes the approach different

We built vector and neural paths, measured them, and rejected them for the final scorer when the evidence did not justify the cost. Equal-weight hybrid retrieval lowered ranking precision at the same candidate depth. The Qwen reranker took about 19.9 seconds for a warm top-10 CPU pass, which would have made a full evaluation impractical.

That led to a metric-driven routing decision: use semantic components where they earn their latency, but select the offline lexical path for the official workload because it scored higher and reproduced reliably.

The final evaluator uses the published deterministic message templates. The Agent applies exact structured updates for those templates, isolates every session, and removes the stale turn-one preference only when the Intent Override arrives. A bounded raw-text fallback remains as defensive behavior for use outside the evaluator.

Challenges

The catalog is large enough that naive retrieval and reranking choices become expensive, but small enough that unnecessary infrastructure can hide the real problem. We had to separate retrieval speed from relevance quality, and implementation complexity from measured value.

The second challenge was evaluation discipline. A feature that sounded more advanced could lower MRR or add hours of CPU work. Preserving every result file and documenting rejected experiments kept the final decision honest.

What we learned

Conversation strategy mattered more than adding another search backend. A good clarification question exposed constraints that the catalog could match, while state and override handling stopped the agent from searching stale intent.

We also learned that semantic similarity and purchase fidelity are different. A product can be semantically related and still violate a disclosed material, color, or budget requirement. The deterministic reranker made those requirements visible in the ranking decision.

Limitations

The deterministic reranker favors literal phrase overlap, which matches the published evaluator but is narrower than unrestricted consumer language. The in-memory BM25 index also has a cold-start build cost, and the 200 public sessions cannot guarantee identical performance on the 800-session final set. A production service would additionally need authentication, rate limits, durable session storage, and monitoring.

Development tools used

  • Python 3.11 and virtual environments
  • Git and GitHub
  • The official TechJam local evaluator
  • JSONL-based datasets and result artifacts

APIs used

  • No external API is used by the submitted Agent.
  • OpenAI-compatible model access and Supabase were explored during development but are excluded from the selected scoring path.

Libraries and frameworks used

  • Python standard library for the submitted Agent
  • FastAPI and Pydantic in the broader experimentation code
  • NumPy
  • PyTorch
  • Hugging Face Transformers
  • Sentence Transformers
  • OpenAI Python SDK
  • Supabase Python client

The selected evaluator policy uses the team's pure-Python BM25 and deterministic reranking code; the model libraries support experiments retained in the repository.

Datasets and assets used

  • Amazon Reviews 2023, Clothing_Shoes_and_Jewelry, through the frozen TechJam catalog
  • 50,000 catalog products
  • 200 public simulated development sessions
  • 800 organizer-held private sessions are described by the challenge but are not present in this repository

Built With

Share this project:

Updates

Submission history