Working on the agent.py

Inspiration

As frequent users of e-commerce platforms for online shopping, we were frustrated when how some e-commerce chatbots were unable to accurately get what we are looking for unless we give them specific instructions and descriptions. Hence, we came together to try to solve this by coming up with our own agent that is able to find products despite ambiguous queries.

How does our agent work?

Intent Routing & Hybrid Pipeline

The agent distinguishes between different forms of shopping intent instead of processing every query with the same retrieval strategy.

At the beginning of each session, it identifies whether the customer is expressing a specific Buying intent, an open-ended Browsing intent, or an intent that may subsequently be overridden. This intent becomes part of the session state and affects how aggressively the system retrieves and recommends products.

The retrieval system uses a multi-route hybrid pipeline. Candidate products are generated using SQLite FTS5/BM25 through several retrieval routes, including: Category matching Keyword matching Product attribute matching Multi-term matching Exact phrase retrieval Previously accumulated conversational constraints Results from these retrieval routes are combined before being passed into a second-stage custom reranker. The implementation describes this as a two-stage system consisting of broad FTS5/BM25 candidate generation followed by direct constraint-aware reranking.

Unlike a traditional BM25-only search engine, the reranker evaluates whether each product actually satisfies what the customer has disclosed. Exact attribute matches, literal phrases, category relevance, price proximity and other signals contribute to the final ranking.

This design aims to improve both Hit Rate@K during candidate retrieval and MRR/Top-K precision during final ranking

Multi-Turn Scenario Evolution

A major component of the system is its conversational state machine. Each shopping session maintains a persistent state containing the customer's category, detailed search terms, identified attributes, budget, previously disclosed constraints, previously recommended products, questions already asked, and other conversational signals.

This allows the agent to perform Information Accumulation. For example, if a customer first requests a jacket, later specifies black, and then specifies a particular material or budget, those requirements are progressively incorporated into the retrieval state rather than treating each message as an independent search query.

The system also explicitly handles Intent Override. Phrases indicating that the customer has changed their mind cause outdated or replaceable constraints to be removed while unrelated constraints are preserved. This prevents obsolete preferences from contaminating future recommendations.

The agent additionally provides proactive clarification. Rather than repeatedly returning low-confidence products when information is insufficient, it can ask targeted questions about: Material Colour Budget Size Style or fit Use case Features Brand Other important requirements

These clarification attributes and their corresponding questions are explicitly defined in the system.

This strategy is designed to reduce unnecessary recommendations and guide the customer toward the correct product within the challenge's 10-turn limit.

Dynamic Context Programming and Runtime Adaptation

The project implements dynamic context programming through a continuously updated session state rather than using a static search query.

Every new user message modifies the active retrieval context. New attributes are accumulated, irrelevant or overridden constraints can be removed, already-recommended products are tracked, and the next retrieval strategy changes according to the information available.

The system also accepts user-profile preference tags during session initialization. These profile terms are maintained separately from stronger session-level constraints so that historical preferences can influence ranking without overpowering the customer's current requirements.

This creates two levels of context: Short-term context consists of requirements disclosed during the current shopping conversation. Long-term context consists of user-profile preference information supplied to the agent.

Importantly, profile information is intentionally given a weaker influence than explicit current-session requirements. This prevents an old preference from overriding what the customer is currently asking to buy.

The agent also dynamically changes its recommendation behaviour according to the session type and conversation turn. For example, vague browsing sessions, high-intent buying sessions, and boundary cases use different recommendation depths and clarification behaviour.

This runtime adaptation forms our implementation of adaptive orchestration: the retrieval and dialog strategy changes dynamically according to accumulated context instead of following one fixed pipeline for every customer.

Optimisation for Product and Efficiency Metrics

The architecture was designed around the challenge's three main evaluation dimensions.

Coverage: Hit Rate@K. Multiple retrieval routes cast a relatively wide initial search net. Category, keyword, attribute, phrase and combined queries help ensure that the target product enters the candidate pool even when the customer's wording differs from the product listing.

Precision: MRR / Top-K Hit Rate. After candidate generation, products are explicitly reranked against the accumulated customer constraints. Products satisfying all disclosed attributes receive additional ranking support, while exact constraint and phrase matches receive stronger scores than generic textual overlap.

Efficiency: Mean Turns to Conversion (MTTC). The agent deliberately controls how many recommendations it produces at different stages of the conversation. When confidence is low, it can obtain more information rather than wasting turns on large recommendation lists. When sufficient constraints have accumulated, it aggressively narrows the candidate set. This balances exploration with early conversion and aims to identify the final purchased Amazon product in as few conversational turns as possible.

How we built it

We built it using VSCode to code and run our code and we used python as our language

Challenges we ran into

Our agent was unable to break the 0.75 hit rate for a long time and after tryout out different solutions and tinkering with the code, we were able to break the 0.75/1 hit rate.

Accomplishments that we're proud of

We are proud that we did not give up when the agent was unable to give up, but instead persevered to look for alternative solutions and finally after hours of discussions and trying out different codes, we were able to improve our hit rate.

What we learned

We learnt to start simple and optimise the code based on failures. We learned that a complicated architecture isn't automatically better. A lightweight FTS5/BM25 system with carefully designed ranking logic could perform well without immediately adding LLMs or embeddings. Changing intent creates subtle bugs. When a user changes one preference, we can't erase everything they previously said. We learned to remove only the outdated constraint while preserving still-valid information.