Bolster

Architecture Diagram
Results of our models

Inspiration

Tiktok Shop e-commerce shoppers discover what new products they want to buy while scrolling their feed. As Tiktok Shop search is keyword-based, the shoppers have no idea what to key in to find the items that they want to purchase. This is because their want was created from a video they saw seconds earlier, but they lack knowledge of the vocabulary related to the item. Queries arrive as an oddly specific situation and intuition. Keyword search assumes the shopper knows the word, but the discovery-led commerce nature of Tiktok Shop produces shoppers that do not.

What it does

Our solution is an agent named Bolster, a headless conversational shopping agent over a frozen 50000-product catalogue. A shopper will talk across it in multiple turns and at every turn it returns ten ranked products. After every turn, if a question is calculated to be able to bring enough value for the subsequent turns, the question will be added as a clarifying question for the next turn.

Our idea is executed by two designs:

We hold the current state of the conversation using a slot dictionary mapping product attributes to values, capturing intent of shopper query. It supports write, overwrite and delete operations. A new English query string is also created from this slot dictionary every turn and a fresh embedding is created from it.
We create a candidate pool from three different streams (semantic, keyword and popularity) unioned in different weights depending on buying and browsing track. We never filter out items as it is an irreversible operation in the pipeline

How we built it

We will be comparing Bolster to a conventional retrieval system in tackling scenarios that are commonly experienced due to our identified problem gap.

They can’t name the product, only the situation In the old system (keyword search), let’s say a shopper types a query like “something for a beach trip”, but a swimsuit listing containing words like “quick-dry”, “UPF 50”, “adjustable straps” has zero overlap with the keyed in words in the query. This swimsuit listing as a result does not surface, or even worse, less relevant item listings start to show up. In Bolster (keyword plus vector stream), every product’s text is encoded into a 384-dimensional embedding. This embedding captures semantic meaning. With a kNN search, this swimsuit will now be able to surface with a “something for a beach trip” despite sharing no words with the query.
They find out what they want by seeing what they don’t: In the old system (accumulated turn embeddings with recency decay): Within the entire conversation, the past turns are encapsulated into a pooled vector. Let’s say we have a buyer that types “black shoes”, then “actually not black, but blue”. The decay never fully removes black from the shopper history, so every turn pulls the wrong candidates into the recall pool and the shopper is forced to constantly see things they have already rejected. In Bolster (slot dictionary with canonical rebuild), we extract relevant fields from the buyer in a slot dictionary that supports write, overwrite and delete. The “actually not black, blue” query deletes black from the slot dictionary and replaces it with blue. A query string is rebuilt from the slot dictionary from scratch every turn. This ensures in future turns that black is absent from every subsequent query.
The catalogue doesn’t have the fields either Even in the event where this problem statement gap is bridged, the catalogue does not have a good coverage value. For example, in the Amazon Dataset given to us, Color Coverage is 4.9%, Material is 4.1% and price is null on 78.9% of rows, so a system that turns a stated attribute into a hard filter removes candidates that could have been the ground truth item. In the old system (structured attribute filtering): When a stated attribute becomes the filter, it takes out candidates that have missing values for the specified field. This runs a heavy risk of the ground truth item being wrongfully eliminated at the start. In Bolster (score, never filter): For every item, we take metadata related to the product (title, description, features, categories, brands and whatever attribute fields exist), concatenated to form one long string. The stated attribute is matched against the product’s full text and will be contributed as a ranking score instead of eliminating it fully from the pool. We run less risk of eliminating the ground truth item early, at most costing the product one feature out of ten.

Impact & Relevance

The gap we identified is a challenge that many Tiktok Shop shoppers struggle with. Around 70% of TikTok users found new brands through scrolling rather than searching specifically for something. This means the scenario where a shopper reaches a search box without knowing what keywords to search for is more common than we think.

TikTok Shop’s global GMV reached roughly $26 billion in the first half of 2025, and US sales are forecast to exceed $20 billion in 2026. At such a huge scale, revenue loss from consumers not buying the products wrongfully filtered out is significant and is not a factor that should be overlooked.

Our project has found the catalogue given from Amazon had poor coverage of metadata, with 4.9% colour coverage, 4.1% material and 78.9% null prices. Yet, with this limited dataset, we found a way to filter on nothing and score on everything. Our project’s architecture is geared towards ranking a catalogue that stays messy and we all know, real catalogues stay messy.

Our solution also complements what has already been released rather than duplicating it. TikTok Shop’s current AI tooling is seller facing: Seller Assistant, List with AI, video generation, aimed at improving how merchants can describe their products, while our product tackles the sparsity issue from demand side, improving how buyers can find the products they want to purchase.

Feasibility & Practicality

Zero API Cost: We do not use LLM calls for our project. Token usage is zero and the main bottleneck would be compute cost rather than token cost. We do have places where we thought of putting LLM into the reranking stage. However, these hooks are instead placed behind flags and are currently disabled. The pipeline is fully functional without them.
Runs on a laptop: The space we consume during the inference process will include 75MB embedding matrix, in-memory SQLite index and plain python dictionaries. All these are in-memory. It takes about five seconds to start and a one-off encoding pass, cached to disk, with retrieval taking roughly 5ms per turn over 50,000 products. We used brute-force exact nearest-neighbour search instead of FAISS or HNSW because scaling an approximate index adds a dependency, build step, tuning parameters and an approximation in exchange for only four milliseconds.
Holds up on a messy catalogue: As mentioned earlier, the dataset given has many items with missing fields. Our system takes note of this and continues to give reliable rankings to each item in the candidate pool. In the event of an empty pool, we will relax the constraint and retrieval repeats. Our design is applicable to merchant-uploaded inventory, which is likely to be even sparser.

Development tools used

VS Code — primary editor
Claude Code (Anthropic CLI) — implementation, review and documentation
Git / GitHub — version control
Python 3.12.1 with venv — environment isolation, pinned requirements.txt
Make — make data, make test, make evaluate as the three entry points
pytest 8.2.0 — 566 tests covering extraction, state, retrieval, ranking, clarification and telemetry
ruff 0.4.4 and black 24.4.2 — linting and formatting
Custom in-repo tooling we wrote: simulate.py (shopper simulator), evaluate.py (fast local scorer), ablate.py (ablation harness), scripts/report_ranker.py (fitted-weight diagnostics)

APIs used

None. Zero external API calls, no LLM, no hosted inference, no vector-database service, no third-party HTTP

Libraries and frameworks used

sentence-transformers 2.7.0 — MiniLM-L6-v2 encoder for the 50k-product embedding matrix and per-turn query embedding
PyTorch 2.2.2 — inference backend for the encoder (transitive; no direct import, no training)
scikit-learn 1.4.2 — LogisticRegression ranker, StandardScaler, GroupKFold validation, ROC-AUC and average-precision metrics
NumPy 1.26.4 — embedding matrix ops, brute-force exact kNN, feature arrays
SQLite FTS5 3.43.1 (stdlib) — full-text keyword index with per-column BM25 weighting, built in memory
pandas 2.2.2 — offline analysis only: weight reports and feature correlations
Python stdlib 3.12 — re, dataclasses, json, collections, statistics, unicodedata

Datasets and assets used

Amazon Reviews 2023 Clothing, Shoes & Jewelry — frozen at 50,000 products (catalog.jsonl, 60MB), supplied by the organiser kit
Public evaluation set — 200 labelled sessions (public_set.jsonl): 80 buying, 80 browsing, 30 intent-override, 10 boundary sentence-transformers/all-MiniLM-L6-v2 — 384-dimensional sentence encoder, used frozen with no fine-tuning
Synthetic conversation corpus (self-generated) — our shopper simulator replays all 200 sessions against the live agent, producing telemetry.jsonl (~77MB, one row per turn with the full candidate pool and ten feature values per candidate)
Ranker training matrix (self-generated) — features.jsonl, 23,012 labelled rows (632 positives), built by joining telemetry against ground truth offline: one positive plus 20 pool-sampled negatives per turn. Labels come from the supplied ground truth, so no hand-annotation was required
Precomputed embedding cache — embeddings.npy, a 50,000 × 384 float32 matrix

Challenges we ran into

Challenge 1 — Changes to the agent could make the ranking model outdated. The ranking model was trained using data generated by our own agent. This meant that whenever we changed the agent’s behaviour or how a feature was calculated, the existing model could become inaccurate without us immediately noticing. We encountered this several times, including one case where improving the category_match feature unexpectedly caused a large drop in performance because the model had been trained using the old definition. From this, we learned that changes to agent behaviour and model training cannot be treated independently. Our solution was to regenerate the training data and retrain the ranking model whenever an important behaviour or feature changed.

Challenge 2 — Evaluation scores were less stable than they initially appeared. We found that running the same version of the system multiple times could produce noticeably different scores, even though the code itself had not changed. This made it difficult to tell whether a new change had genuinely improved the agent or whether the higher score was simply due to variation in the training data. It also showed us that some earlier improvements we had attributed to the agent were actually caused by a more favourable training run. To make our results more reliable, we fixed the source of this variation and saved the trained ranking model so that evaluations of the same version would produce consistent results.

Accomplishments that we're proud of

We built the full end-to-end pipeline and got it working: slot extraction, query reconstruction, intent routing, multi-stream retrieval, clarification, and ranking all running as one pass per turn, with a test suite and an evaluation harness around it. The whole thing runs locally with no external APIs, no LLM calls, and no vector database, 75MB in memory, about 5ms of retrieval per turn across 50,000 products.

What we learned

We tried several ways to narrow down results such as department and category filters, keyword, semantic, and popularity retrieval. The big lesson was that filtering hurts on a sparse catalogue: turning our department filter off improved the score, because hard filters were discarding good products that simply had missing metadata. Scoring on every attribute instead of filtering on any of them is what got us from 0.11 to 0.77.

What's next for Bolster

Currently, Bolster treats every shopping session as a fresh start. It receives a small set of user preferences at the beginning, but anything the shopper does during that session is forgotten once it ends. In the future, we could use a shopper’s past sessions to gradually build a more personalised profile. This would allow returning users to receive better recommendations from the start, rather than having to repeatedly express the same preferences.