Kopi O(1): Conversational E-Commerce Search Agent

A fully offline, stateful shopping copilot that turns vague and evolving preferences into precise product recommendations from a 50,000-item catalog.

Submission item	Link
Public repository	GitHub
Demo video	https://youtu.be/riUw9NpLfrU?is=fcrjVXB_Bzfnkw-t
Track	Track 4 — Shopping Copilot: AI Conversational Search and Recommendations

Inspiration

Customers rarely describe products using perfect catalog keywords. They may begin with a broad request, reveal requirements one at a time, reject a question, introduce a budget, or change their mind halfway through the conversation. A traditional one-shot search query often loses this evolving context.

We wanted to bridge the gap between how people naturally explain what they need and how product catalogs are indexed. Our goal was to build a shopping copilot that could transform an incomplete opening request into the correct target product within at most 10 conversation turns.

What it does

Kopi O(1) holds a multi-turn conversation with a shopper and searches a frozen catalog of 50,000 clothing, shoes, and jewelry products.

On every turn, the agent:

extracts the requested category and constraints such as material, color, size, style, brand, budget, feature, and use case;
distinguishes exploratory browsing from focused buying intent;
saves confirmed preferences in persistent session state;
handles changes of mind by revoking only the superseded constraint instead of clearing the whole conversation;
retrieves and ranks products using everything disclosed so far;
avoids repeatedly recommending the same product; and
asks a useful follow-up question based on the remaining candidate set.

The agent supports the challenge's Buying, Browsing, Intent Override, and Boundary scenarios and returns catalog-valid parent_asin identifiers through the required Agent API.

How our solution addresses the problem

Challenge	Our solution
Vague initial requests	Begin with a broad clarification question before moving to more specific attributes
Preferences disclosed over several turns	Maintain typed, persistent session state
Customers changing their mind	Revoke only the conflicting preference while preserving unrelated confirmed facts
Different browsing and buying behavior	Use separate retrieval and ranking priorities for each intent
A large 50,000-item search space	Combine exact attribute indexes with SQLite FTS5/BM25 retrieval
Numerical budget requirements	Parse maximum, minimum, range, and approximate budgets
Repeated or unwanted questions	Track previously asked and explicitly declined attributes
Scoring that rewards high-ranked hits	Use a deliberate precision-first Top-1 policy before the final turn

How we built it

Kopi O(1) is a deterministic, fully offline Python pipeline composed of five parts.

Conversation understanding and state

The conversation-state tracker converts each message into typed constraints with their normalized value, source turn, and provenance. It recognizes product categories, intent, preference attributes, numerical budgets, declined questions, and intent overrides.

Keeping provenance is important. If a shopper changes one preference, the agent can deactivate only the conflicting slot without losing other independently confirmed information.

Hybrid candidate retrieval

The catalog is loaded into in-memory Python indexes and an in-memory SQLite FTS5 table. Retrieval combines:

exact structured-attribute intersections;
category lookup;
field-weighted BM25 full-text search;
lightweight hand-built synonym expansion;
numerical budget filtering; and
profile-based cold-start candidates.

This lets exact constraints remain strong while still providing fallback candidates when the customer's wording does not exactly match the catalog.

Intent-aware ranking

The ranker blends exact constraint coverage, sequence alignment, hard-slot coverage, category match, lexical overlap, lightweight semantic similarity, budget proximity, profile affinity, product quality, popularity, and recency.

Buying sessions emphasize hard requirements and price fit. Browsing sessions give more weight to semantic relevance, profile affinity, and general product quality.

Adaptive clarification

The dialogue policy estimates the information value of material, color, size, style, brand, budget, use case, and feature attributes over the current candidate pool. It avoids attributes the shopper has already supplied or declined and asks a specific question when that attribute can meaningfully narrow the results.

Precision-first recommendation policy

The current agent sets tail_confidence_threshold = 1.01. Candidate confidence is clamped to [0, 1], so 1.01 is deliberately unreachable. It does not represent 101% confidence.

This setting means:

on turns 1–9, the agent returns only its rank-1 recommendation; and
on turn 10, the threshold is bypassed and the agent returns up to 10 products as a final recall safeguard.

We selected this policy after measuring the trade-off between conversion speed and reciprocal rank on the public development set:

Recommendation policy	Hit Rate@10	MRR	MTTC ↓	Technical score
Strict Top-1 (`threshold = 1.01`)	1.000	1.0000	2.02	0.9796
Confidence tail at ≥0.99	1.000	0.9975	2.02	0.9789
Confidence tail at ≥0.95	1.000	0.9917	2.00	0.9775
Up to 10 products every turn	1.000	0.7110	1.50	0.9033

Showing more products finds some targets slightly earlier, but those hits may appear below rank 1 and reduce MRR. Under the competition's metric weighting, the reciprocal-rank loss is larger than the small efficiency gain. This is an objective-specific engineering choice: a consumer-facing deployment would probably lower the threshold and show a shortlist.

Development tools used

Python 3.10+ for implementation; verified with Python 3.12.5
Git and GitHub for version control and collaboration
the organizer-provided deterministic local evaluator
Python unittest for behavioral and evaluator regression tests
command-line tools for evaluation and experiments

The project is editor-agnostic and does not require Colab, Jupyter, a GPU, or a specific IDE.

APIs used

None. The submitted runtime makes no external API calls.

It does not call an LLM, embedding API, remote search service, hosted database, or cloud platform. It runs without network access or credentials, reports zero prompt and completion tokens, incurs zero model-inference cost, and produces deterministic outputs for the same inputs.

Libraries and frameworks used

The final runtime uses only the Python standard library:

Module	Purpose
`sqlite3` with FTS5	In-memory full-text indexing and field-weighted BM25 retrieval
`re`	Message parsing, tokenization, budget extraction, and intent detection
`json`	Catalog and session JSONL parsing
`dataclasses` and `enum`	Typed session, slot, retrieval, and budget models
`collections`	Inverted indexes and candidate attribute distributions
`math` and `statistics`	Ranking, similarity, proximity, information gain, and query-local confidence
`pathlib` and `unittest`	File handling and automated tests

No pandas, NumPy, PyTorch, scikit-learn, Hugging Face model, vector database, or third-party Python package is required.

Datasets and assets used

We use the competition's frozen package derived from Amazon Reviews 2023, published by McAuley Lab at UC San Diego.

Asset	Details
Category	`Clothing_Shoes_and_Jewelry`
Product catalog	50,000 products in `data/catalog.jsonl`
Product fields	`parent_asin`, title, categories, features, details, description, store, price, average rating, and rating count
Public development set	200 labeled sessions: 80 Buying, 80 Browsing, 30 Intent Override, and 10 Boundary
Private holdout	800 additional sessions retained by the organizer

No additional scraped, purchased, or manually labeled data was added. We use no pretrained model, external embedding set, image, video, or third-party media asset. Direct user identifiers, timestamps, free-text reviews, and raw purchase histories are not exposed to the agent.

Results

On the organizer's released 200-session public development set, the current agent achieved:

Metric	Kopi O(1)	Provided baseline
Hit Rate@10	1.000	0.125
MRR	1.000	0.068034
Mean Turns to Conversion	2.02	9.81
Efficiency	0.898	0.119
Technical score	0.9796	0.10671
Reported model tokens	0	0

All 13 repository tests pass. A complete public-set evaluation, including catalog indexing, took approximately 19.3 seconds on our development machine. Runtime depends on hardware.

These are public development-set results, not private leaderboard results. They do not guarantee the same performance on the organizer's private holdout or on real customer conversations.

Challenges we ran into

Balancing Hit Rate, MRR, and efficiency

Returning more products initially appeared safer because it could find the target earlier. However, lower-ranked hits reduced MRR enough to lower the overall score. Measuring this trade-off led to the deliberate 1.01 threshold and strict Top-1 strategy.

Handling changes of mind

Simply appending every new message to the query produced contradictory constraints when a shopper revised a preference. Typed, individually revocable slots allowed us to retire only the superseded preference while keeping the rest of the conversation intact.

Combining exactness with fallback retrieval

Exact attribute matching is powerful when the customer and catalog use similar wording, but it cannot cover every paraphrase. Combining it with BM25, category routing, and lightweight semantic expansion gives the agent useful fallback behavior without requiring a model or external API.

Accomplishments that we are proud of

Reached every target on the 200-session public development set.
Achieved rank 1 for every public-set hit.
Reduced MTTC from the 9.81 baseline to 2.02.
Improved the technical score from 0.10671 to 0.9796.
Supported buying, browsing, budgets, preference overrides, and boundary responses in one stateful agent.
Kept the final runtime deterministic, fully offline, and free of third-party dependencies and inference cost.

What we learned

Structure can beat scale. Product attributes provide highly precise signals when indexed separately rather than treated only as one text document.
The objective function matters. The best competition policy may differ from the best real-world shopping interface.
State needs provenance. Remembering a preference is not enough; the system must know where it came from so that an override can be handled safely.
Measurement beats intuition. Returning more options appeared safer but performed worse after all weighted metrics were considered.

Limitations

The system was tuned and measured on 200 public sessions and may overfit simulator wording or catalog structure.
Constraint extraction is primarily English and depends on regular expressions, normalized catalog values, and a small handcrafted synonym map.
Unseen paraphrases, spelling mistakes, ambiguous requests, or missing metadata may reduce retrieval quality.
The semantic component uses token expansion and set overlap rather than learned embeddings.
The in-memory FTS5 index is rebuilt at startup and would require persistent indexing at a much larger scale.
The strict Top-1 policy optimizes the competition metric but is less suitable for shoppers who want to compare several options.
The current project provides an Agent API and evaluator workflow rather than a consumer-facing interface.

What's next

Given more time, we would:

build a held-out paraphrase and adversarial test set;
add typo-tolerant and multilingual constraint extraction;
compare offline embeddings and learning-to-rank with the deterministic fallback;
validate confidence and ranking weights on a separate split;
persist the search index and benchmark memory and per-turn latency;
strengthen profile-based personalization; and
build a lightweight interface with a shortlist policy optimized for user satisfaction rather than MRR.