bbq sauce

Inspiration

Shopping online is often a tedious process when it should have been easier without having to physically visit a shopping mall. Hopping onto this challenge would mean that we agree shopping online should not be difficult and is meant to be enjoyable too.

What it does

Shopping Copilot is a multi-turn conversational agent built for the TechJam Conversational E-Commerce Search Challenge. Each session starts with an anonymised customer preference profile and a short opening message. From there, the agent has 10 turns to surface the customer's hidden target product inside a Top-10 recommendation list and asking questions along the way to narrow down what the customer actually wants.

Rather than trying to guess the right product from a vague first message, the agent treats shopping like a conversation. It listens for clues in what the customer says, tracks what it’s learned so far, and decides whether to ask a clarifying question (like material, size, or budget) or go ahead and show recommendations. With every reply, it refines its understanding of the customer’s intent and narrows the product list accordingly.

Under the hood, each turn runs through understanding what the customer said, updating what the agent knows about their preferences, searching the full product catalog for strong matches, and then re-ranking the top candidates to put the most likely product at the front of the list. The agent keeps trying until it finds the target or runs out of turns.

How we built it

We built it using VS Code with the repo's Dev Container / GitHub Codespaces configuration (.devcontainer/devcontainer.json): sets up Python 3.13 (as primary language), installs dependencies, downloads and verifies model weights, and forwards port 8501, all in postCreate. We also used libraries and frameworks such as

• sentence-transformers and transformers (Hugging Face): cross-encoder reranking model
• torch: inference backend, with automatic device selection across CUDA / Apple Silicon MPS / CPU
• scikit-learn, scipy, numpy: scoring, fusion, and numerical utilities
• SQLite FTS5 (via Python's built-in sqlite3): BM25 lexical search index over the product catalog
• Streamlit: the interactive demo UI, including a live "what the agent is considering" debug panel
• huggingface-hub, safetensors, tokenizers: model download, storage, and tokenization infrastructure

The datasets we utilised belong to:

• Amazon Reviews 2023 (McAuley Lab, UCSD): specifically the Clothing, Shoes & Jewelry category, 5-core leave-last-out split, joined on parent_asin. Used for text and structured product metadata only (no images, video, or account data). See DATA_ATTRIBUTION.md in the repo for full terms.
• TechJam-provided competition package: anonymized customer preference profiles and session scenarios used to drive and score the multi-turn dialogue.

Challenges we ran into

• Deciphering the project objectives and narrowing down the specifics.
• We also found it very tedious to adjust the model parameters to produce the results with the highest accuracy.
• Our reranking model’s raw output wasn’t trustworthy on its own, so we had to design a blended scoring approach that combined multiple signals rather than relying on any single model.
• Debugging a black-box ML pipeline where small config changes had outsized, sometimes counterintuitive effects on results.

Built With

• huggingface-hub
• numpy
• safetensors
• scikit-learn
• scipy
• sentence-transformers
• sqlite
• streamlit
• tokenizers
• torch
• transformers