Findly

Inspiration

Search boxes assume you already know what you want. Real shopping doesn't work that way — you start vague, change your mind halfway, and describe things in words the catalog never uses. You say "leather belt with a buckle"; the listing says "full grain cowhide strap, pin closure". Keyword search scores that pair near zero.

We wanted an agent that closes that gap the way a good shop assistant does: by asking one useful question at a time and remembering the answers.

What it does

Findly finds a customer's intended product in a 50,000-item Amazon catalog within ten turns of conversation. Each turn it reads the message, updates what it knows, searches, and returns up to ten products — usually with one clarifying question attached.

It handles the awkward cases: a customer who starts vague, one who states a hard requirement upfront, one who retracts something mid-conversation, and one who genuinely has no preference when asked.

metric	starter baseline	Findly
TechnicalScore	0.1067	0.8153
Hit Rate@10	0.125	0.955
MRR	0.068	0.593
Mean turns to conversion	9.81	3.00

No LLM API, no network access, no third-party packages. Cost: $0.00.

How we built it

A four-stage pipeline, each owned by one person and each measured against a metric:

Understand — extract constraints from the message, accumulate them across turns, detect retractions and erase what they invalidate, and classify what's a hard requirement versus a nice-to-have.

Retrieve — build a query from the customer's exact wording, then BM25 over an in-memory SQLite FTS5 index, 500 candidates deep.

Rank — score every candidate against the full conversation state, penalising hard violations, and cut to the final ten.

Ask or answer — stop asking once we know enough; otherwise pick the attribute worth asking about, chosen by product category.

Every change was measured on 200 development sessions before it shipped. Nothing was merged on intuition.

Challenges we ran into

Our best idea was worth nothing. We assumed the agent was failing because it couldn't find products, so we built a filter to require the customer's must-have terms. It scored exactly zero. We'd measured recall@500 = 1.000 — every target was already in the candidate pool. The agent wasn't failing to find things, it was failing to rank them, and a near-duplicate satisfies the same constraints as the target, so filtering removed only products that were already losing.

Cleaning up the input made things worse. Stripping conversational filler from the search query cost us 0.0065 — because "I don't have a preference for size" contains the word size, and that word was carrying real signal.

Integration was harder than any component. Three people editing one 240-line file produced the same merge conflict four times, and one resolution silently reverted a change worth 0.0155. We caught it by measuring the branch rather than reading it.

Accomplishments that we're proud of

7.6x the supplied baseline, with all four conversation types above 0.90 Hit Rate@10.

Zero dependencies. The entire scoring path is the Python standard library. It runs offline, costs nothing, and starts in twenty seconds.

We measured our failures too. Two complete components — a hard-constraint filter and a dense retrieval route — were built, measured, found not to help, and removed. We can tell you exactly what each was worth, which is worth more than shipping them quietly.

182 tests, and every code review came with numbers rather than opinions.

What we learned

Measure the bottleneck before optimising it. One diagnostic — how deep you must look before the target appears — redirected the whole team from retrieval to ranking and saved us a day of work in the wrong place.

Know your noise floor. Near the end, seven independent ideas all changed the score by less than ±0.002. Once you're there, picking the winner is fitting your test set, not improving your system. We started choosing round numbers and stopping.

A negative result is a result, as long as you measured it.

What's next for Findly

A cross-encoder. Our remaining misses are near-duplicates that keyword scoring can't separate. Rescoring the top 30 with full query-document attention is the direct answer, and the interface is already there.

Learned extraction. Constraint parsing is currently rules and a fixed vocabulary. A small trained sequence labeller would generalise to phrasings we never enumerated.

Personalisation. Each session carries an anonymised preference profile we deliberately left unused — with only 200 sessions we couldn't distinguish a real effect from noise. A larger evaluation set would let us actually validate it.