Inspiration
Traditional e-commerce recommendation systems frequently struggle with conversational dynamics, relying heavily on static keyword matching that fails to capture multi-turn intent evolution, negative preferences, and subtle constraint disclosures.
What it does
Our project addresses these challenges directly by implementing a robust, state-aware conversational agent designed to dynamically bridge the gap between vague, natural-language user queries and a complex catalog.
How we built it
Rather than relying on generic black-box models or pure semantic embeddings—which can introduce noisy "feels-similar" matches at the expense of precise requirements—our architecture follows a disciplined Retrieve → Remember → Rerank → Ask pipeline: Slot & Verbatim Memory Extraction: As the user interacts across multiple turns, the agent extracts both structured slot attributes (such as material, color, size, style, feature, and use-case) and raw, verbatim constraint strings. This dual-layer memory ensures that exact phrases mentioned by the user survive the entire multi-turn session without being lost or corrupted by intermediate interpretations. Additionally, the state tracker gracefully handles "no preference" responses by explicitly marking target slots as answered to avoid repeating questions or adding unnecessary cognitive load.
Deterministic Filtering & Negation Handling: To strictly honor negative constraints (e.g., "not wool or synthetic"), the system incorporates a deterministic exclusion layer. When an explicit negation phrase is detected, candidate products matching the excluded attributes are systematically demoted to the bottom of the candidate pool, preventing invalid options from surfacing during ranking.
Exact-Constraint Reranking: Recognizing that customer disclosures often consist of literal quotes from target product listings, our pipeline evaluates top candidate items against the literal phrases disclosed by the user. Applying an exact-phrase rescorer across an empirically optimized candidate window dramatically improves retrieval precision (MRR) and pushes the exact target product to top ranking positions.
Proactive Follow-up Selection & Defensive Fallbacks: To optimize conversational efficiency and lower the Mean Turns to Conversion (MTTC), the agent evaluates candidate pool overload. It dynamically selects follow-up questions aimed at maximizing information gain based on remaining unfilled slots. If primary attribute slots are exhausted, a defensive fallback tier evaluates secondary dimensions (such as brand, category, or budget) to ensure the system always makes meaningful forward progress.
Challenges we ran into
Handling user boundary cases and infinite clarification loops proved to be significant hurdles. Specifically, recognizing phrased responses like "I don't have a preference" was initially responsible for over half of all session failures by triggering repeated questions. Additionally, optimizing the exact-constraint reranker window required balancing precision against compute costs—wider windows yielded minor metric gains, but paired bootstrap resampling showed performance plateaued beyond a window size of 40.
Accomplishments that we're proud of
We successfully transformed a weak baseline model (stateless BM25 with a HitRate@10 of 0.125) into a highly accurate system achieving a 0.925 HitRate@10, a 0.6345 MRR, and a 0.8022 Technical Score. We are especially proud of designing an agent that accurately narrows candidates down to exact target products in under 4 turns (MTTC of 3.530) purely through structured memory and exact-phrase rescoring.
What we learned
We validated that for task-oriented recommendation systems where disclosures align with product listings, exact literal constraint matching vastly outperforms standard dense semantic embeddings. Counterintuitively, switching to sentence-embedding similarity dropped our performance baseline significantly (down to 0.475), demonstrating that deterministic slot tracking and verbatim constraints provide far stronger precision signals than approximate semantic matching.
What's next for alGAIrithm agent
- Dynamic Question Scoring: Weight follow-up questions by real answer rate (prioritizing attributes answered >90% of the time over those under 5%).
- Secondary Retrieval Path: Build a dedicated candidate recovery path for edge-case products (e.g., generic items like jeans) that structurally miss initial candidate pools.
- Out-of-Sample Window Validation: Test and calibrate the candidate reranking window size against completely unseen validation datasets beyond the initial 200 public sessions.
Log in or sign up for Devpost to join the conversation.