Shopping Copilot

Architectural diagram

Inspiration

Online shopping breaks down the moment you don't know exactly what you want yet. Filters demand precision before you're ready to give it. We wanted to build an agent that behaves like a good salesperson: asking one smart question at a time instead of expecting a perfect search query up front. The TechJam challenge gave us a concrete benchmark to build and measure against, not just a vibe to chase.

What it does

Shopping Copilot is a conversational shopping agent. It reads what a customer says, figures out whether they're actively buying or just browsing, asks one targeted clarifying question when that actually helps, and returns ranked product matches every turn. It handles vague openers, customers who change their mind mid conversation, and shoppers who genuinely have no strong preference at all, without falling apart on any of them.

How we built it

We combined keyword search and semantic embeddings, merged through reciprocal rank fusion so neither signal drowns out the other. A session state machine tracks disclosed preferences across turns, lets them fade if they're not reconfirmed, and wipes them clean the moment a customer changes their mind. An entropy based clarifier picks the single most informative question to ask next. Final ranking blends that retrieval signal with product rating, review volume, and how well each candidate fits everything the customer has said so far.

The whole pipeline runs fully offline and deterministically by default, no API keys or LLM calls needed to reproduce our score. We did build and test three different LLM roles along the way: extracting free text preferences, rewriting search queries, and reranking results. None of them beat the deterministic version once we measured them properly, so they stayed in the code as opt in extras rather than the default path.

Challenges we ran into

Our biggest early bug was a scale mismatch between keyword and semantic scores that silently made half our ranking logic do nothing, worth zero to the final order no matter how we tuned it. Later we found a sneakier version of the same class of bug: one term in our ranking formula could sum up to five times larger than every other term combined, so anything else we tried adding on top of it barely moved the outcome. We only caught it by instrumenting every scoring call across all two hundred sessions and looking at the real distribution, not just the theoretical worst case.

The strangest challenge was realizing the evaluation harness itself gave away more than it meant to. Its simulated customer's opening line always contains the exact category label of the correct product, computed straight from the answer's own listing. We built a version that exploited this and measured a real, meaningfully higher score from it. We chose not to ship it, because it works by reading the test's own answer format rather than actually searching better, and that felt like the wrong thing to optimize for even though nothing forced us to notice or admit it.

Accomplishments that we're proud of

We took our technical score from 0.11 to 0.86 by debugging methodically, changing one thing at a time and verifying every change against the full two hundred session evaluation set rather than trusting a plausible-sounding idea. We tested four separate ways of putting an LLM into the pipeline, measured every one honestly, and rejected all four once the numbers came back worse, which is a much less exciting story than "we added AI and it got better," but felt more honest to actually write down and stand behind.

What we learned

A single aggregate score hides a lot. Breaking results down by scenario type, and eventually down to individual candidate scores, showed us failures a headline number never would have. We also learned that a plausible-sounding idea backed by real reasoning can still fail once measured, more than once, and that finding an exploit isn't the same as the exploit being worth using. Understanding exactly how a benchmark generates its data turned out to matter just as much as the retrieval algorithm itself.

What's next for Shopping Copilot

We'd fold category into the scored retrieval fusion itself instead of only the ranking stage, build out the adaptive orchestration the brief asks for, where the agent switches strategy if a fixed number of turns fail to narrow things down, and try constraining an LLM to only reorder results within a narrow, already-confident band rather than freely rerank everything, to see if that finally gets past what unconstrained reranking couldn't.