mentalist-agent

Inspiration

Online shopping often fails when customers know what they want but cannot express it as a perfect search query. We built Mentalist Agent to recreate the experience of speaking with a thoughtful shopping assistant—one that remembers preferences, asks useful follow-up questions, understands changing intent, and narrows a large catalog without relying on cloud APIs.

What it does

Mentalist Agent is a conversational shopping copilot that searches a catalog of 50,000 products. It maintains context across turns, recommends up to ten products immediately, and asks targeted clarification questions to refine the results.

It can recognize descriptive requests, exact requirements, and intent corrections such as, “Actually, ignore my earlier preference.” Everything runs locally with zero API cost.

How we built it

We combined two complementary retrieval methods:

• SQLite FTS5/BM25 for precise keyword matching.
• A locally hosted BGE embedding model for semantic matches and synonyms.

The results are combined using weighted reciprocal-rank fusion, then reranked using conversation evidence, structured attributes, and small product-quality signals. Query-specific weighting gives semantic search more influence for descriptive requests and less influence when exact or numeric constraints matter.

Precomputed embeddings and deterministic session state make the system fast, reproducible, and fully offline.

Challenges we ran into

The hardest challenge was balancing semantic discovery with exact constraint matching. Semantic search improves recall, but it can drift when a customer specifies a size, material, color, or budget. We addressed this with adaptive fusion weights and soft structured matching.

We also had to handle sparse catalog metadata, vague replies, changing intent, strict ten-turn limits, and reliable offline execution. Another major challenge was ensuring that missing models or invalid artifacts never broke the experience, so we designed a deterministic lexical fallback.

Accomplishments that we're proud of

Mentalist Agent achieved a 94.5% Hit Rate@10 and 0.6456 MRR on the 200-session public development set while reporting zero prompt and completion tokens.

It safely handles multiple independent sessions, supports mid-conversation intent corrections, and connects phrases such as “footwear for jogging” with products described as running shoes.

We also built strict artifact validation, an offline verification workflow, automated tests, and a presentation replay that demonstrates verified success without exposing target products to the agent.

What we learned

We learned that strong conversational search does not require one large generative model. A carefully designed pipeline combining state management, lexical retrieval, compact semantic embeddings, structured evidence, and deterministic ranking can be accurate, explainable, and inexpensive.

We also learned that semantic search works best as a controlled expansion mechanism—not as a replacement for lexical search. Careful query construction and robust fallback behavior were just as important as the retrieval model itself.

What's next for mentalist-agent

Next, we want to learn which clarification question will reduce uncertainty the most instead of following a fixed sequence.

We also plan to:

• Introduce privacy-conscious personalization.
• Improve structured attribute extraction.
• Diversify recommendations across turns.
• Optimize startup time and memory usage.
• Replace exact vector scanning with a scalable approximate-nearest-neighbor index for larger catalogs while preserving offline operation and deterministic fallback behavior.

Built With

• huggingface
• python
• sqlite
• text-embedding