Results
Search Pipeline

AI-MEMORY TARGETTED PRODUCT SEARCH

Project overview

AI-MEMORY TARGETTED PRODUCT SEARCH is a deterministic, stateful and fully offline conversational shopping agent.

It searches a frozen catalog of 50,000 products, returns up to ten recommendations on every turn, and can ask one focused clarification question.

Unlike a stateless keyword search, it remembers the shopper's active intent as the conversation develops and updates that intent when the shopper adds a requirement, rejects an attribute, or changes their mind.

On the public evaluation set, our agent achieved the following results against the provided starter.

Hit Rate@10 of 0.920 (7.36x)
MRR of 0.5245 (7.71x)
mean turns to completion of 3.425 (65% lower)
TechnicalScore of 0.7688 (7.21x)

The Problem & Our Approach

Interpreting the Query

Shopping requests rarely arrive as one complete query. A shopper might...

start with a category...
...then add a budget
then rule out a material...
...and then replace an earlier preference after seeing the first results.

The starter BM25 system searches only the latest message, so it cannot reliably navigate the conversation.

Our agent represents the conversation as structured, revisable state. A typed constraint extractor interprets each message, and a versioned preference ledger records hard requirements, soft preferences, exclusions, removals, and declined attributes. When a customer changes direction, conflicting values are replaced while compatible context is retained.

Search and Ranking

The agent finds candidate products using:

Structured product attributes such as category, material, color, and size.
Keyword search using the shopper's original words (SQLite FTS5).
Keyword search with a small set of related terms (expanded lexical search).
Fallback: A broader category search to avoid missing relevant products.

It then:

Combines results from all search methods (reciprocal-rank fusion).
Removes products that violate requirements or exclusions.
Ranks the remaining products based on how strongly the searches support them, how well they match the shopper's preferences, and whether they align with the shopper's preference tags (Bayesian evidence scoring).

Clarification and follow-up

If too many products still fit the request, the agent asks one question about the attribute that would narrow the list the most. Once the shopper answers or declines the question, the agent records that response and does not ask the same question again.

On later turns, the agent rotates previously shown products to offer new options. If fewer than ten products meet every requirement, it may include near matches found with relaxed search criteria. However, explicit exclusions indicated by the user are never relaxed.

Summary

In summary, our agent is a deterministic, stateful and fully offline conversational shopping assistant. It interprets shopper messages into structured constraints, searches a catalog using multiple retrieval methods, ranks the results, and asks focused clarification questions to refine the search. It runs locally on CPU and requires no external APIs / compute.

Development tools used

Python 3.11+ for the agent, artifact builder, evaluator integration, and experiment tooling.
uv for Python environment creation, dependency locking, and reproducible commands.
Git and GitHub for version control and team collaboration.
Python unittest for automated unit, integration, contract, regression, determinism, and statistical-pipeline tests.
Windows PowerShell and command-line tooling for the verified local build and evaluation workflow.
Claude Code and Codex for the ideation and execution phase, which allowed us to accelerate the pace of exploration and parallel deep dives into all solutions (some didn't work but they gave us clues on the optimal solution)
Cursor for collaborative coding
Cloudflare Workers AI was used while exprimenting with live AI endpoints (not implemented in final solution)

Libraries and APIs used

SQLite as the local artifact store and structured metadata index.
SQLite FTS5 for weighted full-text retrieval across titles, categories, features, details, stores, and descriptions.
Retrieval and ranking methods implemented directly in Python, including reciprocal-rank fusion, Bayesian evidence scoring, information-gain clarification, product rotation, and bounded one-constraint exploration.
No external APIs / hosted vector databases / models are used at runtime. The agent is fully offline and runs the search for 50,000 products on CPU.

Datasets and assets used

Organiser provided catalog of 50,000 products, selected from the Clothing_Shoes_and_Jewelry category of Amazon Reviews 2023.
Organiser provided 200-session public development set