StarGuide - An adaptive conversational shopping agent

Inspiration

Online shopping rarely begins with a perfect query. A customer may start with “I need something for a wedding,” gradually add preferences such as material, style, use case, or budget, and then change their mind halfway through the conversation.

Traditional keyword search often treats every query as a separate request. It may lose useful context, preserve outdated preferences, or return too many loosely related products. We wanted to build a shopping agent that behaves more like a thoughtful store assistant: one that remembers what still matters, recognizes when the customer changes direction, asks useful follow-up questions, and continuously improves its recommendations.

This inspired us to create StarGuide, an adaptive conversational shopping agent that helps users navigate a large product catalog through natural, multi-turn conversations.

What it does

StarGuide is an offline conversational product-search agent that searches a frozen catalog of 50,000 Amazon products and returns up to 10 ranked recommendations in each conversational turn.

The agent can:

understand the customer’s current shopping request;
remember requirements disclosed in previous turns;
distinguish product categories from individual preferences;
recognize when a customer has no preference for an attribute;
detect when the customer changes their mind;
remove outdated preferences while preserving still-valid requirements;
search the catalog through multiple retrieval routes;
combine and rerank product candidates;
avoid repeatedly showing the same products; and
ask a clarification question while still returning recommendations.

This allows StarGuide to support both open-ended Browsing, where a customer is still exploring possibilities, and high-intent Buying, where the customer already has specific requirements.

It directly addresses the problem statement by moving beyond static keyword matching toward context-aware product discovery that adapts as customer intent evolves.

How we built it

StarGuide follows the official Agent.reset() and Agent.respond() interface.

The system processes each conversation through five main stages:

Structured state update: The latest customer message is normalized and added to a session-specific state.
Intent and preference management: The agent identifies active evidence, no-preference replies, and changes in intent.
Multi-route retrieval: The system searches using the complete active query, product category, individual requirements, and a low-weight user-profile route.
Candidate fusion and reranking: Results are combined through reciprocal-rank fusion and reranked using exact evidence matches, query-term coverage, category coverage, and a small product-quality prior.
Recommendation and clarification: The agent returns a fresh Top 10 recommendation list and asks for more information when clarification would help.

The catalog is indexed in memory using SQLite FTS5 and searched using BM25-based full-text retrieval. A bounded LRU cache stores up to 256 retrieval states, preventing memory usage from growing indefinitely across sessions.

Development tools

We used:

Visual Studio Code for coding and project management;
Python 3.10+ as the implementation language;
macOS Terminal for running tests and evaluations;
Python unittest for automated state, ranking, API, and evaluator tests; and
the official local evaluator for reproducible performance measurement.

Libraries and frameworks

The final implementation uses only the Python standard library. Its main components include:

SQLite FTS5 through sqlite3 for in-memory full-text indexing;
BM25 for lexical product retrieval;
json for reading JSONL catalog and evaluation data;
re for text normalization and intent-pattern detection;
pathlib for file-path management;
collections and native Python data structures for state management, candidate fusion, and caching; and
unittest for automated regression testing.

APIs

StarGuide does not use OpenAI, Gemini, Claude, or any other external API.

It requires no API key, hosted language model, third-party machine-learning framework, or network connection during evaluation. Prompt and completion token usage is therefore zero.

The agent implements the official competition interface through:

Agent.reset() for initializing a shopping session; and
Agent.respond() for processing each message and returning a clarification, requested attribute, recommendations, and token-usage information.

Datasets and assets

We used the official competition assets:

a frozen catalog of 50,000 products from the Clothing, Shoes and Jewelry category of the Amazon Reviews 2023 dataset;
200 labelled public development sessions covering Buying, Browsing, Intent Override, and Boundary scenarios;
the official local evaluator;
the official Agent API contract;
the official scoring configuration; and
the official competition documentation.

The organizer retains another 800 sessions for private evaluation.

We did not modify the official evaluator or public labels, and we did not use private evaluation data. The catalog was verified against the official SHA-256 checksum before final evaluation.

Challenges we ran into

One of our largest challenges was handling customers who changed their mind.

If we cleared the entire conversation state, the agent forgot useful information that was still valid. However, if we retained the complete conversation, an explicitly revoked preference could continue to contaminate later search results.

We addressed this by developing a provenance-aware state model. Product categories, independently disclosed requirements, no-preference responses, and revoked preferences are tracked separately. When an Intent Override occurs, StarGuide removes only the obsolete requirement while preserving still-valid context.

Another challenge was balancing recall and ranking quality. A broad search finds more possible targets but may place the correct product too low. A narrow search improves precision but can miss relevant products. We combined multiple retrieval routes with deterministic reranking to balance these objectives.

We also encountered products with very similar categories and feature descriptions. Identifying one exact product can be difficult when a conversation contains no distinguishing evidence. We addressed this where possible through exact feature indexing, term coverage, category coverage, reciprocal-rank fusion, and lightweight product-quality signals.

Finally, some experimental changes improved the public score for the wrong reason. One version retained a preference that the customer had explicitly revoked. Although this produced a slightly higher public-set score, we rejected it because it violated the intended meaning of the conversation and was unlikely to generalize reliably to the private evaluation.

Accomplishments that we're proud of

StarGuide achieved the following results across all 200 official public sessions:

Metric	Result
Hit Rate@10	0.9900
Mean Reciprocal Rank	0.723849
Mean Turns to Conversion	2.380
Efficiency	0.862
Technical Score	0.884555

Its scenario-level performance was:

Scenario	Hit Rate@10	MRR	MTTC
Buying	0.9875	0.663036	1.8875
Browsing	1.0000	0.777584	2.2000
Intent Override	0.966667	0.756508	4.0000
Boundary	1.0000	0.682500	2.9000

Compared with the official starter, StarGuide substantially improved product coverage, ranking quality, and conversational efficiency.

We are especially proud that the final solution:

finds the target product within its Top 10 recommendations in 99% of public sessions;
performs strongly across all four scenario types;
runs entirely offline during evaluation;
requires no paid model or external API;
has zero marginal API cost;
keeps customer-session information local;
uses only the Python standard library;
passes all automated regression tests; and
reproduces its results using the official, unmodified evaluator.

These public-set results are development and reproducibility signals. The organizer’s private 800-session evaluation remains the authoritative final assessment.

What we learned

We learned that conversational product search is not only a retrieval problem. How an agent represents evolving customer intent can be just as important as the search algorithm itself.

We also learned that forgetting too much and remembering too much can both cause failure. Effective conversational state management requires the agent to preserve valid context while selectively removing information that the customer has explicitly replaced.

Another important lesson was that two systems with similar overall scores can still make very different mistakes. Our earlier implementations had complementary strengths: one offered stronger recall and Intent Override performance, while the other provided stronger early ranking. The final version combines their useful mechanisms into one agent instead of running two agents as an ensemble.

Finally, we learned to prioritize semantic correctness and expected private-set robustness over a higher score caused by a public-set artifact. A technically higher number is not necessarily a better result if the underlying behavior contradicts the customer’s request.

What's next for StarGuide - An adaptive conversational shopping agent

StarGuide currently relies primarily on lexical retrieval. A future version could add an offline semantic-retrieval route to improve its understanding of synonyms, broader scenarios, and requests that do not share exact words with catalog descriptions.

We also plan to improve the clarification strategy. The current agent requests additional information efficiently, but a future system could analyze the current candidate pool and ask the question expected to reduce uncertainty the most.

Other potential improvements include: