Browse2Buy

Demo Video

Watch our Browse2Buy demo here: https://youtu.be/pCHXWQzG8K0

Inspiration

Product search often breaks down when users do not know exactly what they want. Browse2Buy connects broad exploration with more focused product search, helping users gradually refine their preferences and move toward confident product decisions through conversation.

What it does

Browse2Buy is a conversational shopping copilot with two adaptive retrieval modes:

Browsing focuses on discovery, semantic recall, and diverse recommendations.
Buying focuses on precise matching against explicit user constraints.

The system tracks preferences across turns, handles preference overrides, asks clarification questions when the search space is still too broad, and adapts the retrieval strategy as the user's intent becomes more specific.

How we built it

We first annotated ~50k products into structured semantic attributes such as category, brand, color, material, features, and use cases using a self-hosted Qwen3.8-27B model. The same self-hosted Qwen3.8-27B also interprets each user turn to extract shopping constraints and preference changes.

For the Buying path, we use BGE-small-en-v1.5 to semantically expand user constraints and SQLite FTS5 BM25 for field-weighted lexical retrieval.

For the Browsing path, we use Qwen3-Embedding-0.6B over structured product representations for dense semantic retrieval, alongside an independent BM25 lexical route. The two ranked lists are combined using Reciprocal Rank Fusion (RRF) and diversified using Maximal Marginal Relevance (MMR).

For clarification, we use a candidate-aware mathematical scoring strategy based on candidate coverage, expected search-space reduction, answerability, and remaining-turn value. This allows the agent to choose the question that is most useful for narrowing the search.

All models are self-hosted, with no external model APIs.

Development tools used: Visual Studio Code, Python CLI, Git/GitHub, local browser-based debug UI, PowerShell/Git Bash.

APIs used: Local OpenAI-compatible API endpoint running Qwen3.8-27b for catalog annotation and turn interpretation. No external production APIs are required.

Libraries and frameworks used: Python standard library, SentenceTransformers, Hugging Face Transformers, PyTorch, NumPy, SQLite FTS5/BM25, HTML/CSS/JavaScript.

Datasets and assets used: 50,000-product catalog, 200-session public evaluation set, 400-session Manual400 benchmark, V5 attribute annotations, canonical attribute dictionary, BGE-small attribute embeddings, and Qwen product-card embeddings.

Challenges we ran into

The main challenges were maintaining correct dialogue state across multiple turns, handling preference overrides without losing valid constraints, balancing lexical precision with semantic recall, and deciding when the agent should ask another question instead of immediately returning recommendations.

We also had to keep the Buying and Browsing pipelines sufficiently different so each could optimize for its own retrieval objective.

Accomplishments that we're proud of

Our system achieved a 0.915 Hit@10 rate, a 0.7441 technical score, and an average latency of approximately 4 seconds per conversation turn.

We are also proud of building separate adaptive Buying and Browsing pipelines, candidate-aware clarification, dependency-aware preference updates, and a fully self-hosted retrieval stack.

What we learned

We learned that conversational product search is much more than retrieval alone. Strong performance depends on combining dialogue-state tracking, semantic understanding, lexical and dense retrieval, clarification strategy, and dynamic routing between different search behaviors.

We also found that sparse and dense retrieval are most valuable when they complement each other rather than trying to solve the same problem.

What's next for Browse2Buy

Next, we plan to further reduce per-turn latency, improve the over-generality cutoff so broad queries can skip unnecessary computation, strengthen long-term personalization, and continue optimizing sparse-dense retrieval fusion.