ASTRA (Adaptive State-Tracking Retrieval Agent)

• 

  Intent-Aware Retrieval

• 

  Structured Catalogue Filtering

• 

  End-to-End ASTRA Architecture

• 

  Information-Gain Questioning

• 

  Entropy-Based Attribute Selection

• 

  Adaptive Long-Term Memory

• 

  Relevance-Gated Personalisation

Inspiration

Modern consumer shopping habits can no longer be satisfied by traditional e-commerce search methods, such as static keyword matching. Previously, traditional e-commerce uses static keyword matching, which worked well for simple purchases. However, consumers today are now more informed, selective and overwhelmed than they were before. For purchases over $100, 96% of customers research online whilst 75% spend 1-4 weeks comparing options before deciding. Moreover, consumers today require more guidance in their shopping process - more customers are turning to conversational AI assistants for shopping help to save time and effort. 31% of consumers use AI chat tools in online purchases every time they shop, citing benefits such as higher confidence in recommended products, finding better deals and saving time and effort.

However, current modern e-commerce platforms also do not adequately address the needs of online shoppers. 2 in 5 consumers say generative AI recommendations make the search process more frustrating than traditional search. We narrowed this problem down two three main pain points.

Firstly, a generic agent poorly distinguishes between users that are intent on buying a specific product, versus users that are browsing and require more broad assistance and recommendation from the shopping agent. Someone who is buying would want more focussed, narrow recommendations from the agent tailored to their specific needs; someone who is just browsing would appreciate the breadth given by broader recommendations. However, the reality is that most shop agents lie in an awkward position that satisfy neither user profile.

Moreover, existing shopping agents may ask questions to shoppers that are ineffective. A poorly phrased question such as "Is the colour you're looking for Vantablack" when only a few out of thousands of items are in that colour filters out few items from the search space, which barely narrows down the list of possible items which the user actually wants and takes longer for the user to arrive at a product that they actually want.

Finally, an agent’s poor recommendations cause shoppers to abandon the shopping process. For instance, when agent repeatedly suggests unsuitable products, users stop believing it understands them. The Indeed example showed that filtering bad recommendations reduced unsubscribes by 5%. Moreover, users expect the agent to narrow the search space. If customers still have to manually filter through irrelevant results, the agent provides little advantage over traditional search.

We therefore developed ASTRA: The Adaptive State-Tracking Agent.

What it does

ASTRA is a conversational shopping agent over a 50,000-item Amazon Clothing, Shoes & Jewelry catalogue. Across a session it:

• Detects intent and routes retrieval accordingly. Buyers get hard constraint filtering and tight ranking; browsers get wider semantic exploration and more weight on their long-term taste profile.
• Chooses its next question by information gain, asking the attribute that prunes the candidate space most aggressively rather than an arbitrary one.
• Gates its output. Recommendations below a confidence threshold are suppressed: ASTRA asks another question instead of guessing. A separate relevance gate keeps a user's long-term profile from contaminating an unrelated search.

How we built it

In-session state

Every conversation turn begins with an LLM call that updates a structured state object:

{
  "intent": "buying",
  "category": "shoes",
  "hard_conditions": {
    "price_max": 80,
    "department": "women",
    "min_avg_rating": 4.4,
    "store": "nike"
  },
  "disclosed_attributes": {
    "size": 7,
    "material": "leather"
  },
  "negated": ["heavy", "black"],
  "asked_attributes": ["material", "size", "colour"]
}

Tracking asked_attributes is what stops the agent repeating itself; tracking negated is what stops it re-suggesting things the user already rejected.

Keyword Matching Layer

Regex parsing and tokenisation feed an in-memory SQLite FTS5 index over the catalogue. We run a strict term1 AND term2 AND ... query first; if it returns fewer than 30 candidates we fall back to an OR query.

Results are ordered by SQLite's BM25 with field weights we tuned by hand:

• title 6.0
• categories 4.0
• features and details 2.5
• store 1.5
• description 1.0

They are then rescored in Python on term coverage, popularity and a retrieval-position prior. An exact keyword hit bypasses embedding entirely, which is the cheapest path through the system. Otherwise, it falls to the next layer.

Categorical Filtering Layer

We audited every catalogue column for population rate, cardinality and structure before deciding what was safe to filter on. features and description are unstructured free text and belong to the embedder.

categories has approximately 880 entries after cleaning with heavy semantic overlap, so hard filtering on it throws away valid items.

In the end, categories which persisted included:

• price
• department (11 mutually exclusive canonical values after cleaning)
• average rating
• rating count
• store

Embedding Layer

We fine-tuned BAAI/bge-base-en-v1.5 (768 dimensions) on 800 hand-generated (anchor query, positive product, hard negative product) triplets using multiple-negatives ranking loss, for 3 epochs at batch_size=8, gradient_accumulation_steps=4.

Training loss fell from 0.58 to 0.14.

We benchmarked against OpenAI text-embedding-3-small and our local model won on both retrieval quality and latency. Catalogue vectors are pre-computed and cached to disk, so per-turn retrieval is one matrix product.

Long-term memory

A user's taste is a single L2-normalised vector v₂, loaded at session start and frozen for the session. The current session state is embedded as v₁.

Before using memory at all, we check a relevance gate:

$$ \cos(v_1, v_2) > \tau $$

If it fails, memory is switched off. If it passes, each catalogue vector pᵢ is scored as:

$$ s_i = a\,(p_i^\top v_1) + b\,(p_i^\top v_2) $$

with a > b for buyers (trust the stated request) and b > a for browsers (lean on taste).

Because the weights are scalars, we fold them into the query first and compute:

$$ p_i^\top(a v_1 + b v_2) $$

— one inner product per item instead of two.

At session end we extract only positive, reusable preferences and embed them as vₙₑw. Rather than update the profile at a fixed rate, we let the session decide how much it deserves to move things:

$$ \text{sim} = \cos(v_2, v_{\text{new}}), \qquad \alpha = \tfrac{1}{2}\,(1 - \text{sim}) $$

$$ v_3 = \text{normalize}\big((1-\alpha)\,v_2 + \alpha\,v_{\text{new}}\big) = \frac{(1-\alpha)v_2 + \alpha v_{\text{new}}} {\lVert (1-\alpha)v_2 + \alpha v_{\text{new}} \rVert_2} $$

The learning rate is half the cosine distance between what we already believed and what we just saw. A session that only confirms the existing profile barely moves it; a session that contradicts it moves it a lot.

The ½ caps α at 0.5 for any non-negative similarity, so no single session can outvote accumulated history. The useful consequence is that the profile self-stabilises: as it converges on a user's taste, similarity rises, α falls, and updates shrink — without us tracking a session count or hand-tuning a decay schedule.

Renormalising matters more than it looks. Both inputs are unit vectors but their weighted sum is not — its length depends on the angle between them, so an erratic user's profile would quietly shrink, weakening both the b(pᵢᵀv₂) term and the relevance gate. Renormalising keeps every comparison on the same scale.

We deliberately do not persist transcripts, budgets, negations or session categories.

Entropy-based querying

Over the top 100 candidates C, for each candidate attribute A with values v:

$$ H(C) = -\sum_{v} p(v)\log_2 p(v) $$

$$ H(C \mid A) = \sum_{v} p(v)\,H(C_v) $$

$$ IG(A) = H(C) - H(C \mid A) $$

We scale IG(A) by how well-populated the attribute is across the candidates, so a column that only 5% of items fill can't win on a technicality, and ask the user about the two highest-scoring attributes.

Concretely: with 100 items split 50/50 by department, "men's or women's?" halves the space whichever way it's answered. An example which best illustrates this is as follows: "Is it silk?" with 2 silk items eliminates 2 items 98% of the time and wastes a turn.

Confidence gate and generation

After ranking, the top 10 picks are taken and their similarity is measured to the session state. Anything below the threshold gate of 0.3 is dropped; if all ten fail, ASTRA returns no recommendations and asks another question.

A second LLM call (local Llama 3.1 via Ollama) packages the two chosen attributes into a natural question.

Model choices

State tracking and intent detection run on the DeepSeek V3 API for quality; response generation runs locally. If the network drops, both fall back to Ollama, so the demo never dies on a bad connection.

Challenges we ran into

Firstly, brand identity was ambiguous. The catalogue has both store and manufacturer, and they disagree on 13.29% of items. We compared them across all 50k rows: 82.29% match exactly, 86.08% after case normalisation, and the divergences are mostly sub-line extensions (Skechers vs Skechers for Work), corporate suffixes (Victorinox vs Victorinox MFG), distributor IDs, and outright multi-brand OEM noise (Jerzees listed under Hamilton Beach). store is the better brand proxy, and there are zero rows with a store but no manufacturer.

Moreover, sparse columns broke naive entropy. An attribute can look maximally informative while being present on almost nothing, which produced confident, useless questions. The coverage weighting was the fix.

Memory leaked across contexts. Our motivating failure case: a professional diver whose ten previous sessions are all dive gear, now shopping for a gift for his mother. Unweighted long-term memory drags every result toward men's technical equipment. The relevance gate exists because of this case, and tuning τ meant measuring the actual distribution of pairwise cosine similarities across catalogue items to find where "related" stops and "coincidence" starts.

Users with intent override also posed a major issue. When a shopper overrides their earlier intent, stale constraints have to be revoked without losing anything they might re-state later. We advance a search epoch (which clears the seen-items set, since previously rejected products may now be valid), mark conflicting constraints revoked, and rebuild the active query terms from scratch from live slots every turn. That rebuild-from-scratch design means a re-mentioned preference is instantly active again on the very next turn.

Finally, latency. Entropy over the full candidate set, two embedding passes per item, and two API round trips per turn was too slow. Restricting entropy to the top 100, folding the memory weights into a single inner product, pre-caching catalogue vectors, and moving response generation to a local model brought it back to conversational speed.

Accomplishments that we're proud of

Our fine-tuned embedder beat a commercial API. On 800 hand-built triplets and three epochs of training, bge-base-en-v1.5 outperformed OpenAI's text-embedding-3-small on our retrieval task at lower latency and runs locally, for free.

Most conversational agents ask follow-ups an LLM felt like asking; on the other hand, ours picks the attribute that maximises information gain over the live candidate set, which means we can explain and defend every question the agent asks.

We designed from a data audit, not from intuition. The store-vs-manufacturer comparison across all 50,000 items, the population-and-cardinality pass over every column, and the pairwise cosine similarity distribution used to set the relevance threshold each changed a design decision we'd otherwise have made by guessing.

Local model fallback means ASTRA still works with no network. The confidence gate means it withholds rather than hallucinates. The seen-items filter means it never repeats a recommendation within a search epoch. None of these are visible when everything goes right, which is the point.

The whole pipeline runs at conversational speed over a 50k catalogue on modest hardware, after the latency work described above.

What we learned

The interesting problem in conversational shopping relates to dialogue policy. A strong embedder with a bad question strategy still takes ten turns to find a jacket. Framing question selection as information gain turned a vague design argument into something we could compute and defend.

We also learned to audit data before designing around it. Half our architecture decision came out of a column-by-column look at the catalogue rather than from intuition.

Moreover, the confidence gate and the relevance gate both make ASTRA say less. Both make it more trustworthy, because the failure mode that loses a shopper isn't a missing recommendation, it's a confident wrong one.

What's next for Adaptive State-Tracking Retrieval Agent (ASTRA)

Continual fine-tuning of the embedder on accumulated user prompts, richer long-term profiles that separate style from colour from store affinity rather than collapsing them into one vector, and a proper offline evaluation harness measuring turns-to-target against a ground-truth item.

Built With

• baai-bge
• deepseek
• fts5
• json
• llm
• ollama
• python
• pytorch
• sqlite
• transformers
• vector