ExactMatch Shopper

Inspiration

In the race to build conversational AI, there is a massive over-reliance on generative models for tasks they weren't designed to do. Probabilistic LLMs introduce latency, hallucination, and unpredictable variance. Drawing from my studies in data science and business analytics, alongside my hands-on experience engineering automated data pipelines using SQL and Python, I learned that production-grade systems require rigid, deterministic constraints. You simply cannot build a reliable application if the underlying data architecture is guessing.

I wanted to bring that enterprise-grade pipeline stability to e-commerce. The goal for this TechJam was to build a Shopping Copilot that treats the LLM not as the core search engine, but purely as a semantic safety net operating over a lightning-fast, zero-token deterministic lexical baseline.

What it does

ExactMatch Shopper is a high-speed, hybrid conversational shopping agent that reliably routes complex user intents across a 50,000-item catalog. It tracks what a shopper reveals turn-by-turn, detects whether they are buying with a strict requirement or just browsing, and adapts gracefully when they change their minds mid-conversation.

Unlike standard agents that immediately ping an LLM, ExactMatch defaults to a zero-cost deterministic engine. It only triggers its AI semantic reranker when it detects a vague edge case or boundary condition, resulting in an ultra-low Mean Time To Cart (MTTC) without sacrificing precision.

How we built it

The copilot operates on a Dual-Track Hybrid Retrieval Architecture, orchestrated through Python and robust SQL logic:The Deterministic Engine (Zero-Token Baseline): Instead of relying on slow vector databases, I engineered an in-memory SQLite FTS5 (Full-Text Search) matrix. This handles rapid lexical matching and hard-filtering deterministically. The agent tracks conversation state via Slot Accumulation and features greedy Slot Decay—automatically relaxing constraints if the structured filters yield zero results.Variance-Based Questioning: To drive the conversation efficiently, the agent dynamically calculates the entropy of missing attributes within the candidate pool. Relying on statistical computing principles, the system only asks the user about attributes that will mathematically divide the remaining search space.The Fail-Open LLM Semantic Sniper: Once the SQLite engine isolates the top candidates, a lightweight LLM API (integrating endpoints like Groq or OpenAI) acts as a reranker. Crucially, this is a fail-open architecture. If the API is rate-limited or disabled, the application degrades gracefully to the SQL baseline.The performance of the system is evaluated on Hit Rate at 10.

Challenges we ran into

The hardest challenge was the Boundary Scenario—conversations where users give vague or "no preference" responses. Initially, strict lexical filters caused the hit rate to tank to 0.40 in these edge cases.

I solved this by introducing Boundary Anchor Fallbacks. When a user gives no preference, the agent dynamically injects baseline conversational anchors to protect recall. We then unleash the LLM reranker exclusively to untangle the semantic ambiguity. This surgical use of the AI pushed the boundary hit rate back up to a balanced 0.715.

Accomplishments that we're proud of

The system's metrics across the 200-session public evaluation set speak for themselves:

Zero-Token Mode (No LLM): Achieved a massive 0.690 (highest at 0.695) Hit Rate@10 with an ultra-fast MTTC of 5.42, proving the sheer power of an optimized SQLite pipeline.

LLM-Powered Mode: The semantic reranker pushed the overall Hit Rate to a peak 0.710 (Technical Score: 0.575) while maintaining a strict 5.82 MTTC.

Total Stability: The pipeline processed 252 AI rerank attempts with zero crash errors, proving the fallback logic is completely stable under load.

Token Usage even with LLM was not too high with average total_tokens being around 200k per run.

What we learned

This project fundamentally reshaped how I view AI orchestration. I learned that bridging raw machine learning statistics with modern LLM endpoints creates a far more resilient product than using either in isolation. It shows me that LLM can indeed be to a certain extent be deterministically evaluated and audited so as to improve efficiency and token usage while improving accuracy. I also learnt how to audit and evaluate LLMs in chatbots to see where fixes need to be done and what is being done right or wrong by the LLM and whether my tokens were a waste or indeed useful.

What's next for ExactMatch Shopper

The next step is to transition the in-memory SQLite FTS5 architecture into a persistent, distributed database environment to handle catalogs exceeding a million SKUs. Furthermore, I plan to refine the variance-based questioning logic into a more advanced reinforcement learning loop, allowing the agent to continuously self-optimize its clarification strategies based on real-world conversational telemetry. I also wish to use this concept in the future to evaluate my LLM conversational bot projects to improve efficiency and clearly trace the LLM's impacts.