WOW-WILDLOTUS Shopping Copilot

fun video: https://www.youtube.com/watch?v=LV8hPPqcsGI

Inspiration

Online shopping search often fails when customers cannot describe exactly what they want, provide incomplete requirements, or change their minds halfway through a conversation.

Traditional search engines treat every query independently, while many conversational shopping systems return a fixed list of products without reasoning about what to ask next or how many products to show.

We built WOW-WILDLOTUS Shopping Copilot to make product discovery adaptive.

Instead of treating recommendation as a one-shot retrieval problem, WOW-WILDLOTUS continuously updates its understanding of the shopper and dynamically decides:

What should we ask next?
How many products should we recommend right now?

What It Does

WOW-WILDLOTUS is a multi-turn conversational shopping agent that searches a catalog of 50,000 products for a hidden target item.

During each conversation, the agent:

extracts product categories and structured hard and soft constraints from natural language;
maintains an isolated intent state for every session;
detects whether the user is refining, browsing, buying, or replacing an earlier intent;
retrieves and ranks products using structured matching, BM25, hybrid recall, and weighted rank fusion;
asks an informative clarification question about attributes such as material, color, size, feature, or use case;
dynamically chooses whether to show 0–10 products, instead of always returning a fixed Top 10; and
adapts when the customer provides no additional preference or changes their mind.

The required competition output remains fully deterministic and evaluator-compatible:

Message → structured ask_attribute → ranked catalog parent_asin values

How We Built It

Our production pipeline consists of four main stages:

Understand → Intent Router → Retrieve & Rank → Dynamic Slate

1. Understand

A local Qwen 3.5 4B model, served through Ollama, converts the current user message into grounded product categories and typed attribute slots.

Importantly, new evidence is first stored as a turn-level observation rather than immediately modifying the committed shopping intent.

This separation allows later stages to decide whether the new information represents:

an ordinary refinement,
an additional preference,
a partial intent change, or
a completely new shopping goal.

Deterministic grounding and regex fallbacks protect the pipeline when model output is missing, malformed, or invalid.

2. Intent Router

The Intent Router decides whether new evidence should be accumulated into the existing shopping intent or should replace part or all of it.

It supports:

explicit intent overrides;
partial and full intent replacement;
removal of incompatible stale constraints; and
different routing strategies for focused Buying behavior versus exploratory Browsing behavior.

This prevents outdated preferences from contaminating later recommendations when the shopper changes their mind.

3. Retrieve & Rank

Products are retrieved from a read-only 50,000-item catalog indexed with:

SQLite FTS5
a preprocessed structured-attribute sidecar

The retrieval pipeline combines multiple complementary signals:

Strict constraint matching + Lenient matching + BM25 lexical search + Structured scoring + Raw-text recovery + Weighted reciprocal-rank fusion

Strict matching improves precision when catalog metadata is reliable.

However, strict filtering alone can accidentally eliminate the correct product when metadata is missing or an attribute is misunderstood. Therefore, lenient and hybrid retrieval paths provide additional recall and prevent incomplete catalog fields from becoming hard failure points.

4. Decide with Dynamic Slate

Our Dynamic Slate planner jointly decides:

which clarification attribute to ask about next, and
how many products to recommend at the current turn.

Instead of always returning a fixed Top 10, the planner evaluates the utility of an immediate hit against the expected value of obtaining more information from future user responses.

It uses a two-observation lookahead aligned with:

Hit Rate
Reciprocal Rank
Turn Efficiency

This allows WOW-WILDLOTUS to return a small, precise slate when confidence is concentrated, while widening the slate when additional coverage is more valuable.

Interactive Diagnostic Interface

We also built a Chainlit diagnostic interface using the same Agent implementation as the headless competition evaluator.

The interface provides:

interactive product cards;
a visual pipeline graph;
per-node production traces; and
step-through evaluation.

This makes the agent's behavior inspectable rather than treating the recommendation pipeline as a black box.

Performance

On the 200-session public evaluation set, our default live local-NLU pipeline achieved:

Metric	Result
Technical Score	0.783
Hit@10	93.0%
MRR	0.584

These are public development results and are not claims about performance on the private test set.

Latency, Token Usage & Estimated Cost

The following measurements describe the default local-NLU configuration. Actual performance depends on CPU, GPU, memory, and whether the model and indexes are already warm.

Item	Local `qwen3.5:4b` Pipeline	Regex Fallback
Warm per-turn latency	~25–35 seconds due to sequential Ollama calls	SQLite retrieval and CPU planning only
First startup	Several minutes for preprocessing, index construction, and model loading	Index construction only
Router token usage	~200–1,500 prompt / 10–80 completion tokens	0
Understand token usage	~2,500–6,000 prompt / 150–500 completion tokens	0
Paid API cost	US$0	US$0
Runtime network dependency	Localhost Ollama only after initial download	None

A live-NLU evaluation covering approximately 400 conversation turns uses roughly:

1–3 million prompt tokens
0.1–0.3 million completion tokens

including the Understand stage.

The current respond()["usage"] field reports Router token usage only. Understand-stage token usage is measured and disclosed separately because it is not currently included in that response field.

All default inference runs locally, resulting in zero paid API cost. The remaining runtime cost is local electricity. For the 200-session public evaluation, we estimate this to be well below US$1, depending on the machine and local electricity price.

The optional semantic reranker also runs locally and is not included in the reported token usage.

Challenges We Faced

1. Incomplete and Noisy Product Metadata

A strict intersection of every extracted constraint can accidentally remove the correct product when a catalog field is missing or an attribute is misunderstood.

We addressed this with:

Strict + Lenient candidate pools → Hybrid recall → Weighted rank fusion

This preserves precision without allowing imperfect metadata to become a single point of failure.

2. Refinement vs. Intent Change

A major conversational challenge is determining whether a new message adds information to the existing request or represents a completely different shopping goal.

Simply accumulating every message causes stale preferences to contaminate future recommendations.

Our Intent Router therefore separates turn-level observations from committed session state and supports both partial and full intent replacement.

3. How Many Products Should We Show?

Returning ten products increases coverage but may produce an early hit at a poor rank, hurting reciprocal-rank quality.

Returning only one product can provide excellent ranking quality when correct—but risks missing too much of the catalog.

Dynamic Slate explicitly models this trade-off instead of relying on a fixed Top-K rule.

4. Local LLM Reliability and Latency

Local language models introduce latency and occasional formatting errors.

To make the system robust, we implemented:

bounded retries;
grounding checks;
deterministic fallbacks;
catalog preprocessing;
index caching; and
reproducible setup scripts for Windows, macOS, and Linux.

Accomplishments We're Proud Of

One stateful Agent, two interfaces — the same implementation powers both the official evaluator and interactive web demo.
Robust Intent Override — handles changing user goals without relying on public scenario labels or known target IDs.
Dynamic Slate Planning — jointly selects the next clarification question and recommendation count.
Local-first architecture — requires no paid API and can operate without internet after the model and catalog are downloaded.
Inspectable production traces — visualizes which pipeline branches actually execute during each turn.
Strong public evaluation performance — 0.783 Technical Score, 93.0% Hit@10, and 0.584 MRR on the 200-session public set.

Limitations

Our current implementation still has several limitations:

Local-model latency: Sequential Ollama calls make live NLU slower than a purely deterministic system.
Hardware dependence: Inference time varies significantly across CPU, GPU, Apple Silicon, and available memory.
Run-to-run variation: The local LLM may produce slightly different structured observations across repeated runs.
Approximate future planning: Dynamic Slate uses a two-observation lookahead with catalog-signature and response-model approximations rather than full counterfactual retrieval.
Probability calibration: Candidate probabilities, parser uncertainty, no-preference likelihoods, and posterior tail mass could be calibrated further.

What We Learned

The biggest lesson was that conversational recommendation is not only a retrieval problem.

A strong shopping agent must jointly reason about:

Intent State + Catalog Uncertainty + Question Selection + Ranking Quality + Recommendation-List Size

We also learned that LLMs are most effective when used for uncertain language understanding, rather than as an unverified final product judge.

In WOW-WILDLOTUS, the local model interprets the conversation, while catalog retrieval, ranking, planning, and exact product-ID evaluation remain transparent and deterministic.

What's Next for WOW-WILDLOTUS

Next, we plan to:

improve probability calibration for Dynamic Slate;
train stronger attribute-answer and no-preference models;
reduce local-NLU latency;
expand robustness testing with paraphrased and imperfect-English buyer agents;
evaluate lightweight semantic reranking; and
explore richer personalization.

Our goal is to improve recommendation quality while preserving the system's core principles:

Local-first. Adaptive. Reproducible. Inspectable.