innovation 1
innovation 5
best result
innovation 2
innovation 3

🐱 TwoMeow — Conversational E-Commerce Search Agent

Inspiration

Online shopping search works well when users already know exactly what they want. Real conversations, however, are rarely that clean.

A customer might begin with a vague request such as "I'm just browsing," gradually reveal constraints like "I need waterproof running shoes under $80," suddenly change direction with "ignore what I said earlier," or simply say "use your own judgment."

With a catalog of 50,000 products and at most 10 conversational turns, we realized that the core challenge was not simply retrieval. It was deciding:

What is the most useful thing to ask next — and when should we stop asking?

This inspired TwoMeow: a conversational search agent that treats every dialogue turn as an evidence-gathering decision rather than following a fixed sequence of questions.

What It Does

TwoMeow is a multi-turn conversational e-commerce search agent designed to identify a user's target product efficiently from a large catalog.

Instead of asking predefined clarification questions, TwoMeow continuously analyzes the current candidate pool and selects the attribute that provides the most useful information.

At every turn, the agent can:

understand new or changing user preferences;
distinguish positive preferences from exclusions and "no preference" responses;
retrieve and rerank products from the full 50,000-item catalog;
decide which clarification question is most informative;
detect when further clarification is no longer useful; and
return a ranked Top-10 recommendation list.

The default submission runs fully offline, keeps its search structures in memory, and uses zero LLM tokens.

How We Built It

Each respond() call executes a six-stage pipeline:

1. Intent Router

The agent classifies the current interaction as:

Buying
Browsing
Intent Override
Boundary

Override signals are detected explicitly so that conflicting historical preferences can be cleared before the next retrieval step.

2. Slot Tracker

User responses are converted into structured constraints.

Importantly, TwoMeow distinguishes between:

positive preferences;
negative/exclusion preferences; and
no-preference signals.

This prevents statements such as "I don't want leather" from accidentally sending leather into retrieval as a positive keyword.

3. Hybrid Retriever

Dynamic-field BM25 provides the primary retrieval path.

Dense retrieval separates product information into two semantic spaces:

Identity: title, categories, store
Attributes: features, details, description

Dense retrieval is activated lazily through a risk gate only when lexical retrieval appears weak.

4. Clarifier

For every remaining attribute, TwoMeow computes an information score based on:

$$

\text{ClarificationScore}(a)

\text{Coverage}(a) \times \text{Entropy}(a) $$

The attribute with the highest useful information content becomes the next clarification question.

When all scores fall below the early-stop threshold

$$ \tau = 0.3 $$

the system stops asking low-value attribute questions and switches to a wildcard question instead.

5. Ranker

The top candidates are reranked using:

dynamic attribute weights;
confirmed-slot field coverage;
profile priors; and
risk-aware MMR.

Dynamic attribute importance is recomputed every turn:

$$ w_a = \text{EvidenceConfidence} \times \text{Recency} \times \text{PoolSelectivity} \times \text{AttributePrior} $$

The weighting is applied during both the 50,000 → 100 and Top-20 → Top-10 stages.

6. Response Builder

Finally, the agent produces the official response contract:

{recommendations, ask_attribute, message}

This allows TwoMeow to recommend products, ask a clarification question, or do both within the same turn.

Core Innovations

🧠 Entropy-Driven Clarification

TwoMeow does not follow a fixed question order.

Instead, clarification attributes are selected using real-time coverage × entropy over the current candidate pool.

When the candidate pool becomes small, local entropy is blended with global statistics from the 50,000-product catalog to reduce noise.

⏹️ Early Stop

🚫 Negation-Aware Purification

Each user response is decomposed into:

positive constraints;
exclusion constraints; and
no-preference signals.

Negative terms are stored separately and applied as ranking penalties rather than being mistakenly treated as positive retrieval terms.

🔀 Risk-Gated, Field-Separated Dense Retrieval

Dense retrieval separates identity information from product attributes and fuses them at query time.

However, semantic retrieval is not always beneficial.

TwoMeow therefore bypasses Dense retrieval for Buying queries and whenever BM25 already produces a strong, category-stable candidate set. This prevents semantic retrieval from introducing unnecessary noise.

⚖️ Candidate-Pool-Aware Dynamic Weights

The importance of a confirmed attribute is not fixed.

TwoMeow recomputes its weight every turn using evidence confidence, recency, candidate-pool selectivity, and an attribute prior.

As the conversation evolves, ranking priorities evolve with it.

🔄 Atomic Intent Override Reset

When a user says something like "ignore my earlier preference," stale constraints can seriously damage subsequent retrieval.

TwoMeow detects this pattern explicitly, clears conflicting slots atomically, and prevents old raw query text from leaking into the new search context.

Challenges We Ran Into

1. Asking more questions sometimes made performance worse

Our initial intuition was that collecting more attributes would always improve ranking.

In practice, once the candidate pool had already converged, additional questions often added little information while increasing the number of turns.

This motivated the entropy-driven clarification strategy and eventually the early-stop mechanism.

2. Semantic retrieval was not always better than lexical retrieval

Dense retrieval improved semantic matching in some difficult cases, but we found that it could also introduce unrelated candidates when BM25 had already found a strong lexical match.

Instead of treating Dense retrieval as universally superior, we introduced a risk gate that activates it only when it is likely to help.

3. Negation is surprisingly dangerous in product search

A response such as "anything except leather" contains a highly relevant product term — leather — but retrieving products using that word produces exactly the opposite of the user's intention.

We therefore separated positive evidence, exclusions, and no-preference responses before retrieval and ranking.

4. Intent changes can contaminate the entire conversation state

Simply adding a new preference was not enough when users explicitly overrode previous requirements.

Old constraints and raw query text could continue influencing retrieval even after the user had changed their mind.

This led us to implement an atomic intent-reset mechanism.

5. Public-set performance did not automatically imply generalization

Our strongest public-set configuration achieved a large improvement over the official baseline. However, a separate blind experiment showed a much smaller gap.

Rather than hiding this result, we treated it as an important lesson: optimizing a conversational retrieval system requires careful separation between benchmark performance and true generalization.

What We Learned

Building TwoMeow changed how we thought about conversational search.

The biggest lesson was that retrieval quality alone is not enough. A successful shopping agent must also manage uncertainty and decide how to spend a limited interaction budget.

We learned that:

the best clarification question depends on the current candidate distribution, not a predefined script;
knowing when not to ask can be as important as choosing what to ask;
lexical and semantic retrieval are complementary rather than universally interchangeable;
explicit handling of negation and intent overrides is critical in multi-turn search;
evaluation must consider both ranking accuracy and conversational efficiency; and
strong benchmark results should always be tested against held-out data before making claims about generalization.

These lessons shaped TwoMeow from a retrieval pipeline into a more adaptive conversational decision system.

Tech Stack

Layer	Technology
Language	Python 3.10–3.12
Lexical Retrieval	SQLite FTS5 (in-memory, BM25)
Dense Retrieval	sentence-transformers · `all-MiniLM-L6-v2`
Fusion	Reciprocal Rank Fusion (RRF)
Reranking	Local field-aware scoring + risk-aware MMR
LLM (Optional)	Anthropic Claude — disabled by default; zero tokens in submission
Configuration	PyYAML
Numerics	NumPy
Testing	pytest
External Database	None — all data structures are kept in memory

Evaluation Results

We evaluated TwoMeow on the official 200-session public evaluation set using:

PYTHONHASHSEED=0
BM25-only default configuration
zero LLM tokens

Metric	Official Baseline	TwoMeow
HitRate@10	0.125	0.955
MRR	0.068	0.707
MTTC	9.81	2.865
Efficiency	0.119	0.814
TechnicalScore	0.107	0.852

TwoMeow substantially improves both ranking quality and conversational efficiency on the public evaluation set.

Generalization Check

We also evaluated the system on a separate 400-ASIN blind test with zero overlap with the public set.

System	TechnicalScore
Plain BM25	0.761
TwoMeow	0.751

This result is important: although TwoMeow achieves strong gains on the official public sessions, the held-out experiment does not yet demonstrate a generalization improvement over plain BM25.

We therefore treat the public-set improvements as experimental results rather than evidence of proven generalization.

What's Next

TwoMeow demonstrates that conversational product search can be framed not only as a retrieval problem, but also as an information acquisition problem.

Future work could focus on improving the generalization of dynamic clarification and retrieval gating, validating the approach on larger unseen session sets, and learning clarification policies directly from interaction outcomes.

Our ultimate goal is simple: