SHOPFRENS

Inspiration

We wanted to build a system to make product discovery easier. Instead of users having to search through a list of the same products, our system will allow them to search more specifically for what they want. We hope that our project will be able to help users find what they're looking for more effectively and efficiently.

What it does

This project implements a lightweight conversational search shopping agent for recommending products from a product catalogue. The agent is designed to support multi-turn conversations rather than treating each user message as an independent search query. It maintains session-level state, accumulates useful terms from the conversation, extracts budget constraints, retrieves relevant products using SQLite FTS5, and asks targeted clarification questions to progressively refine the user's preferences.

The implementation is fully offline and has no LLM dependency. It uses an in-memory SQLite FTS5 virtual table to index product information such as each catalog product's title, categories, features, details, store and description. A separate product_meta table holds parsed prices for budget filtering. Products within the user's budget are prioritised, although products above budget can still appear if insufficient alternatives are available.

User messages are tokenised, common stopwords are removed, and useful terms are accumulated across turns. Up to 40 terms are retained. SQLite's BM25 ranking is used to identify relevant products, with different weights applied to product fields. A candidate pool of 50 products is considered. The agent will asks follow-up questions about attributes such as category, material, colour, budget, size, style and intended use. Clarification stops after a a predefined cutoff.

As there is no API or LLM inference cost, the direct computational cost is very low. Hence the only costs that need to be accounted for is computing, storage, development and maintenance as well as infrastructure. For a small to medium catalogue, this should be considerably cheaper to operate than a system that calls a hosted LLM for every user interaction.

Although the implementation provides a simple and deterministic conversational shopping pipeline, there are several areas that could be improved. The agent relies on regular expressions and tokenisation rather than semantic language understanding. For example, budget extraction is based on a fixed collection of regular-expression patterns. This means that more complex expressions, implicit preferences, negations, synonyms, and nuanced user requirements may not be interpreted correctly. The retrieval system is also lexical. Products that use different terminology from the user's query may not rank highly even if they are semantically relevant. If we had time to fix this issue, we would use an LLM or a dedicated natural-language understanding component to convert each user message into structured preferences and constraints. It may also be possible to introduce embedding-based retrieval or a hybrid BM25 + semantic retrieval system. Another main issue is that accumulated terms are treated mostly independently. The search query is constructed by OR-ing accumulated terms. Consequently, the system does not explicitly model relationships between preferences. A method that could be used to improve this would be to extract structured positive and negative constraints and apply them separately during retrieval and ranking.

How we built it

Our offline conversational agent is built entirely in Python using the standard libraries with no external LLMs or APIs required. Our goal was to build a reliable search assistant under these core ideas:

Natural Language Parsing: We built a smart language parser that filters out words and conversational fillers. This allows our agent to pinpoint the exact keywords that matter such as extracting the price limits e.g., "under $50"
Stateful Context & Memory: Our SessionState class cleans user messages, removes filler words, and remembers up to 40 search terms across turns to refine recommendation results
Targeted Clarification: Our agent asks about missing details in a prioritised order and stops asking questions after turn 7 to focus on refining the final recommendation

Challenges we ran into

Parsing Language Without an LLM: Without an LLM, capturing subtle context, indirect requests, or synonyms through simple keyword matching is difficult. While our parser extracts explicit details such as price caps fairly well, relying on regex and tokenisation naturally limits how deeply our agent understands nuanced conversations
Fine-Tuning Parameters: Finding the right balance for arbitrary parameters such as setting the BM25 field weights to favour titles over long descriptions, or capping the search terms at 40, required endless trial and error. Without a clear path, fine-tuning the balance between accuracy and conversion speed was a major guesswork game

Accomplishments that we're proud of

Firstly, our agent operates with zero LLM dependency and is built entirely from scratch using only Python's standard library and SQLite for the ultimate speed and zero API overhead. Secondly, our system packs all multi-turn tracking, parsing, and search logic into a single clean file while still scoring 0.795HR@10, 0.486MRR, and 0.669 overall on the 200-sample test set. Lastly, our agent delivers intelligent context and budget management, seamlessly remembering up to 40 search terms across turns while instantly parsing price caps like "under $50" to surface the most relevant, budget-friendly matches.

What we learned

Through this project, we learnt that building a recommendation system is not just about finding matching products. In order to cater to each users unique preferences, it is important to understand their needs, ask the right questions, and continuously adapt recommendations based on the information they provide.

We also learned that a simple retrieval-based system can be surprisingly effective, but there is a clear trade-off between simplicity and semantic understanding. This helped us understand where traditional search works well and where AI-based approaches could provide additional value.

What's next for SHOPFRENS

Our next step is to make SHOPFRENS more semantically intelligent by adding an LLM or a dedicated natural-language understanding component. This would allow us to evolve from a product search agent into a personal shopping companion that will be able to genuinely provide use and convenience to users.