JOFC2.0

How we built it We implemented RouteWise as a Python agent using the competition’s required Agent.reset() and Agent.respond() interface. Each customer message first passes through a deterministic conversation-state system. It extracts constraints, detects exclusions and intent overrides, and remembers what has already been asked or recommended. The search layer builds an in-memory SQLite FTS5 index over product titles, categories, features, details, brands, and descriptions. It runs several weighted BM25 searches for the overall query and individual attributes. Reciprocal-rank fusion combines these candidate lists. The resulting products are then filtered and reranked using required constraints, preferred attributes, exclusions, budget, exact feature phrases, ratings, and a small popularity tie-break. We also developed an optional local MiniLM CrossEncoder. MiniLM does not search the catalogue or generate product IDs. It can only reorder the top 20 products already selected by the deterministic system, and only for sufficiently detailed requests. Safety gates preserve strong deterministic results and automatically fall back if the model is unavailable or uncertain. Challenges we ran into The greatest challenge was balancing recall and precision. Strict filtering could remove the correct product when catalogue metadata was incomplete, while broad retrieval could find the product but place it too low. Multi-turn state created another challenge. The agent had to distinguish between a preference, a requirement, an exclusion, and a complete change of intent. Phrases such as “not leather” represent exclusions, while phrases such as “will not fade” describe positive product features. We also discovered that adding a model does not automatically improve a system. Semantic reranking helped richly specified searches but could reduce ranking quality for broad browsing requests while adding CPU latency. This led us to keep deterministic retrieval authoritative and use MiniLM only behind conservative gates. Finally, we had to avoid overfitting to the 200 public sessions. Public results are useful regression evidence, but they cannot guarantee performance on the private evaluation set. Accomplishments that we're proud of On the 200 public development sessions, our deterministic agent achieved: Hit Rate@10: 1.000, compared with the supplied baseline’s 0.125 MRR: 0.626341, compared with 0.068034 MTTC: 1.735 turns, compared with 9.81 Technical Score: 0.873202, compared with 0.106710 Model usage: zero prompt and completion tokens We are especially proud that RouteWise handles all four evaluation scenarios—Buying, Browsing, Intent Override, and Boundary—while returning only valid catalogue IDs. The current implementation also passes 88 automated tests and remains completely functional without MiniLM, model downloads, paid APIs, or network access. What we learned We learned that a reliable AI shopping system needs a strong retrieval foundation before it needs a language model. Multi-route BM25, explicit state, carefully guarded filtering, and measurable evaluation produced most of our improvement. We also learned that clarification and recommendation should happen together. The agent can ask for a useful detail while still providing immediate results, allowing it to find the correct product earlier. Most importantly, every component should have a constrained role. Deterministic code owns state, catalogue access, filtering, and product IDs. MiniLM is only allowed to improve the ordering of an already-grounded shortlist. What's next for JOFC2.0 Our next priority is improving first-rank precision for broad browsing requests while preserving the current Hit Rate@10. We would also like to: Build larger category-held-out and paraphrased test sets that better represent private evaluation risks. Improve handling of missing prices and incomplete product descriptions. Measure and reduce catalogue-indexing and MiniLM inference latency. Package an approved local MiniLM checkpoint for fully offline semantic evaluation.

Built With

• bm25
• minilm-l6
• python-3.11
• sqlite-fts5