Shopping Copilot — SCOPE Agent

Inspiration

Online shopping offers endless choice, but finding the right product is still frustrating. Shoppers rarely express every requirement in one perfect query. They refine their budget, add must-have features, change colors or materials, and sometimes switch to a completely different product category.

We wanted to build more than another search box. We wanted a shopping copilot that could remember evolving preferences, recognize when the user changes direction, ask useful follow-up questions, and return recommendations at every turn.

That idea became SCOPE: Stateful Conversational Offline Product Engine.

What it does

SCOPE is a multi-turn conversational product-search agent designed for a frozen catalog of 50,000 clothing, shoe, and jewelry products.

For each conversation, SCOPE maintains an isolated session state containing information such as:

product category;
size, color, and material;
budget;
intended use;
must-have features;
additional free-text requirements.

The agent recognizes explicit intent changes and removes outdated preferences instead of accumulating contradictory constraints. It also handles responses such as “no preference” without converting them into false product requirements.

After every user message, SCOPE returns a ranked Top 10 product list. When additional information would be helpful, it can also ask one focused clarification question in the same response, subject to a maximum clarification budget of four questions.

The submitted configuration does not require an external LLM, API key, hosted vector database, or network connection during inference, provided that the local MiniLM model and product embedding index have already been prepared.

How we built it

We implemented SCOPE as an offline-first Python retrieval and ranking pipeline.

Conversation state management accumulates preferences across turns and detects explicit intent overrides.
SQLite FTS5 with Porter stemming provides weighted BM25-style retrieval for precise titles, brands, categories, and product attributes.
Local all-MiniLM-L6-v2 embeddings provide semantic retrieval for use cases, paraphrases, and less exact descriptions.
Weighted reciprocal-rank fusion combines BM25 Top 160 and Dense Top 200 candidates.
Constraint-aware reranking classifies catalog evidence as MATCH, VIOLATION, or UNKNOWN.
Exact-phrase scoring rewards products containing clearly disclosed user requirements.
A conservative hard-constraint gate removes a product only when the catalog contains clear contradictory evidence.

BM25 has a fusion weight of 1.0, while Dense retrieval normally uses 0.25. After an explicit intent override, the Dense weight is reduced to 0.15 because a newly reset query contains less context and is more vulnerable to semantic noise.

The resulting pipeline is:

User message
    → update conversation state
    → generate clarification question when useful
    → BM25 Top 160 + MiniLM Dense Top 200
    → weighted reciprocal-rank fusion
    → constraint-aware reranking
    → conservative hard-constraint gate
    → Top 10 recommendations

The repository also contains a guarded OpenAI-compatible adapter for future experimentation. It is disabled in the submitted SCOPE configuration and was not used to produce the reported results.

Challenges we ran into

The hardest challenge was handling evolving intent. A shopper might initially request black running shoes and later say, “Actually, I need watch repair tools.” SCOPE must recognize this as a category change, clear the outdated shoe preferences, and rebuild the search query around the new request.

Another challenge was distinguishing missing information from an actual mismatch. Product metadata is often incomplete. Treating every unknown attribute as a violation would incorrectly remove potentially relevant products, so SCOPE separates UNKNOWN evidence from confirmed VIOLATION.

We also had to balance keyword precision and semantic recall. Dense retrieval can recognize paraphrases and use cases that BM25 misses, but it can introduce noise—especially after an intent reset. We addressed this with weighted fusion and a lower Dense weight after explicit overrides.

Clarification planning created another trade-off. The agent has a limited question budget, so it must ask about useful missing requirements without delaying recommendations. SCOPE therefore returns recommendations immediately while prioritizing focused, high-value questions.

Finally, the semantic model and 50,000-product embedding index must be built or packaged before offline evaluation. Once those artifacts are available locally, the submitted inference path requires no external service.

Accomplishments that we're proud of

We evaluated SCOPE and the provided baseline implementation with the unchanged evaluator and identical fixed split IDs from the 200-session public set.

On our fixed 40-session public validation split, SCOPE substantially outperformed the baseline:

Metric	Baseline	SCOPE
Hit Rate@10	0.1750	0.9500
MRR	0.1178	0.5546
MTTC — lower is better	9.325	2.825
Technical Score	0.1563	0.8049

On the 160-session development split, SCOPE achieved:

Hit Rate@10: 0.9250
MRR: 0.5942
MTTC: 3.2438
Technical Score: 0.7959

Across all 618 agent responses in the complete 200-session public set, the average measured response latency was approximately 42.9 ms, excluding one-time model and index initialization.

These results were produced without external model calls, prompt tokens, completion tokens, or model API cost. The private organizer evaluation remains the final test of generalization.

What we learned

We learned that conversational shopping is not only a retrieval problem—it is also a state-management problem. Remembering what changed, what still applies, and what should be removed can be as important as understanding the latest message.

We also learned that sparse and dense retrieval are complementary. BM25 provides precision for names, brands, categories, and exact attributes, while embeddings provide flexibility for paraphrases and use cases. Constraint-aware reranking is then necessary to keep the final recommendations faithful to the shopper’s stated requirements.

Another important lesson was that missing product information should not automatically count as failure. Explicitly representing MATCH, VIOLATION, and UNKNOWN made the system more conservative and prevented incomplete catalog fields from eliminating otherwise relevant products.

Most importantly, we learned that an AI-style conversational experience does not always require a hosted generative model. Careful retrieval, state tracking, and ranking design can provide fast, private, reproducible behavior locally.

What's next for Shopping Copilot — SCOPE Agent

Given more time, we would improve the handling of complex negation, comparisons, and implicit preference changes. We would also add multilingual conversation support and clearer explanations showing why each product was recommended.

For ranking, we would explore a compact local cross-encoder or learning-to-rank model, evaluate it on a larger held-out dataset, and learn calibrated retrieval and constraint weights rather than tuning them manually.

We would also package the local MiniLM model and embedding index as a documented release asset so the complete Dense retrieval path can be reproduced in an offline scoring environment with minimal setup.

A guarded LLM adapter may eventually assist with unusually complex language understanding, but product retrieval, constraint enforcement, and final ranking would remain grounded in the frozen catalog.

Our goal is simple: make SCOPE feel less like a search engine and more like a personal shopping copilot that remembers, adapts, and responds to what the shopper actually needs.