Pretaste

AI agents guess who will buy what they make. Pretaste checks those guesses against Qloo's taste data before anything is made, then rewrites the listing where Qloo disagrees.

Comment

AI agents now plan, write and ship products, but they decide who the buyer is by guessing. Pretaste is the grounding step an agent runs before it makes anything: each checkable buyer guess is tested against Qloo's taste data on the fans of comparable titles, and the listing is rewritten where Qloo disagrees.

  • What it fixes: an agent's guess about the buyer, made before any product exists.
  • What only Qloo gives it: recorded values on the real fans of comparable titles, so the agent's own guesses can be overturned, held or traced. The LLM assumed buyers of an everyday-Japan coloring book want a cozy look; cozy is not among the 386 style tags Qloo ties to the fans of its comparables, so the look is marked Challenged and the rewrite leans minimalist, offered as a version to test against the original (cozy is also a common search word). Where Qloo has no value, the guess stays Not established instead of sounding sure.
  • The honest result: across nine sealed test sets a blind judge still preferred the LLM alone in most runs (below). Even where Qloo's coverage is strongest, the data corrects the buyer picture but does not yet make the decision clearer, and we show why.

In the gallery run 2 of the 5 planned comparables were not found in Qloo, so the checks rest on Midnight Diner, Convenience Store Woman and Before the Coffee Gets Cold, a show and two novels rather than coloring books. Their fans' strongest style tags are Minimalistic, contemplative, Visual-Rich and Visually Vibrant; travel is not a Qloo style tag, so it was left unchecked rather than guessed. The screen says only "Minimalistic" came from Qloo and the rest is the writer's wording. The female lean (+0.45) supports the gender guess; the guessed age bands sit no higher than the others, so age stays not established.

Inspiration

We gave an AI agent a small online business to run. It could plan, write and ship, but when it came to what buyers would like, it guessed. Its first Etsy printable was rejected by our founder on sight ("looks like an internal corporate presentation"), and earlier social posts were thrown out for the same reason: the agent had decided who the buyer was without any evidence.

So we built the missing step. Before the agent makes anything, its picture of the buyer is checked against real taste data, or clearly marked as unchecked.

What it does

You give Pretaste one to three product ideas, the marketplace (Etsy, Amazon KDP or other) and, optionally, a price in USD and your own audience assumptions.

  1. The agent guesses first. An LLM-only pass writes down seven guesses about the buyer (age, gender, life stage, aesthetic, what else they like, trend direction, US region) and the listing it would ship without taste data. This is the "before".
  2. The agent picks comparables. For each idea it plans two or three: a bestselling book or major brand on the product's own subject, and popular titles whose fans clearly do that activity. When the product is a kind of book people use (a coloring book, journal, log or planner), the first pick must be a bestselling title of that same kind, because a novel that only shares the setting does not describe who buys coloring books. Picks the agent itself marks as off-subject are dropped before any lookup, so an idea can end up with one. If Qloo finds fewer than half of an idea's comparables, the agent searches up to two other titles once; those count as planned too, and the misses stay listed and still count.
  3. Qloo describes the comparables' fans. Pretaste finds each comparable in Qloo and asks who its fans are, which style tags they lean to, what else they like, where in the US the signal is stronger, whether it is trending, and how much the ideas' taste tags overlap.
  4. Fixed rules judge, not the LLM. Each guess is marked Supported, Challenged or Not established by thresholds in code:
    • Age: the guessed age groups are compared with the other age groups of the same audience; a gap beyond 0.05 counts as a lean. If the guessed groups lean as a whole but one of them is more than 0.05 below the others, the verdict is mixed and stays Not established.
    • Gender: the female skew must pass ±0.05.
    • Aesthetic: each guessed look is matched to a Qloo style tag, then ranked among every style tag Qloo ties to those fans. The strongest quarter counts for the guess; the weaker half, or missing from the full list, counts against it. A look that is not a Qloo style tag is not checked.
    • Stability: if leaving out any one comparable would flip an age or gender verdict, or only one comparable gave the signal, it becomes Not established and the screen names the comparable it rests on.
    • Life stage has no Qloo signal, so it always stays Not established and the screen says why.
    • Your own notes ("bought mostly by teenage girls") are split into age and gender claims and checked against the fans of the comparables for the idea the note is about. A note that claims a majority can be contradicted but never confirmed, because affinity is not a buyer share. When Qloo cannot check a note, the LLM gives its general-knowledge read under the label "General knowledge, not checked against data".
  5. The agent writes the "after". A new listing (title, tags, description) and a research priority for each idea: Research first, Revise, then research, Lower research priority, or Hold · more evidence needed. Search tags that repeat a look Qloo challenged are dropped. When the data cannot separate the ideas (a tie, or another idea with too few comparables found to be ranked), every one is held, and a separate panel, "Suggested first", labelled general knowledge, names one idea to start with and why. A practical problem that stops an idea whatever the data says (the marketplace can't make it, a rights risk) is shown as "Check before you start".
  6. A checker guards every Qloo claim. A sentence presented as Qloo evidence must cite a value the server recorded (same call, entity, field and value) or it is downgraded to "interpretation" or removed. Affinity can't be turned into a purchase percentage, and a price claim must match the price you entered.

The result opens with a summary: how many of the checkable guesses Qloo challenged and how many could not be checked, the reason to rethink each one with the value behind it, Fans also like (the titles Qloo ties most strongly to those fans, offered as ad targeting and comparison keywords, not as proof of buyers), and the before and after drafts with the new words marked. Five tabs follow: Assumptions, Before & after (with a switch that hides the Qloo version), Ideas ranked, Audience and Evidence. Any value chip opens the recorded call behind it. Runs can be cancelled, restarted, and saved as Markdown with every Qloo call (parameters redacted).

Agents can call it directly: the repository ships an MCP server with one tool, pretaste_check, that runs the same pipeline and returns the verdicts, the ranking and the rewritten listing.

How we use Qloo

Guess or check Qloo path What it answers
Comparables exist /search Turns names into Qloo entities; unrelated or ambiguous matches are rejected
Age, gender /v2/insights (demographics) Whether the fans score higher in the guessed age groups than in the others, and which way gender leans
Aesthetic /v2/tags, then /v2/insights (style tags, paged) The Qloo style tag for each guessed look, and its rank among all the style tags tied to those fans (up to 1,000)
What else they like /v2/insights (neighbors, explainability) Taste neighbors and which comparable drove each; another volume of a comparable's own series is left out
US region /v2/insights (heatmap) US states and territories where the signal is stronger
Trend /v2/trending Whether comparables that are artists, brands, movies, podcasts or TV shows are rising. Books are not covered; when no idea has trend data, ideas are ranked on affinity alone and the screen says so
Idea vs idea /v2/analysis/compare Share of taste tags two ideas have in common

Without Qloo, every one of these is the model's own assumption. With it, each verdict points to a recorded call you can open.

Our own decisions, before and after

We took three real decisions for businesses our AI agent runs, copied the "before" from our existing records, froze it in the repository (docs/cases/), and then ran each one on the live app. These runs used an earlier version (Oct 24) with an older ranking formula; the second case was re-run on a later version and the decision did not change. Two of the sample themes had already been seen during development; the case files say which.

Decision LLM alone said Pretaste said What we did
Etsy printable shop: second product Make the pet-care sheet now Within 3 points of the product we already sell (96 vs 93, first called a tie through a rounding bug since fixed; 95 vs 94 on the re-run), too close to call; an off-niche trend printable had no taste evidence Kept our plan: the 30-day traffic check on the first listing comes first; dropped the trend printable
KDP coloring book: next edition Make the third idea Also ranked the third idea first, but the gap came from which comparables the agent picked, not from Qloo (two ideas were the same book yet got different comparables) Did not act on either; keep the current book until the 30-day data
LINE stickers in Japan: next set Skip a new set and promote the two released ones Not applicable: US taste data can't show Japanese sticker demand Kept our plan (a winter set for the same receiver)

None of these has sales results yet; we check them again at our 30-day reviews (Nov 17 and 18).

How we built it

  • Next.js, React, TypeScript and Tailwind on Vercel. A run is 11 steps called one by one so the screen shows progress; a full run with live Qloo takes about a minute (55 seconds in the gallery run).
  • A typed Qloo client: GET only, hackathon host only, allow-listed paths only; retries on 429 and 5xx and records every call for the Evidence tab. A repeat of the identical request within 7 days reuses that live answer (kept in memory only) and is labelled "cached"; the gallery run made 39 calls.
  • Name matching rules: a comparable resolves only on an exact name, a listed alias, the first entry of a series, the same title with a subtitle (no sequel marker), or one record that is clearly more popular than every other record with the same name. Anything else stays unresolved instead of guessed.
  • Claude (Anthropic API) writes the language steps, parsed against schemas. Verdicts and the research priority are computed in code: (0.4·affinity + 0.3·trend) ÷ 0.7 − 0.2·tag similarity, shown out of 100; without trend data it is affinity − 0.2·tag similarity. An idea is ranked only when at least half of its planned comparables were found in Qloo; that share is a gate, not part of the score. Ideas less than 8 points apart are a tie (re-running the same input moved a score by up to 7 points in our tests). When trend data exists, a clear leader is re-ranked under nearby weightings to show whether it stays first.
  • Mock mode runs without keys (marked "SYNTHETIC, not Qloo data").
  • CI runs a leak check, type check, lint, 85 regression tests and the build on every push.

Built with AI coding tools, directed by a founder who is not a software engineer. The decisions above are ours.

Challenges we ran into

  • Silently ignored parameters. A wrong parameter doesn't error; it returns something else, so we check every response for the fields we asked for.
  • Names to entities. Titles with many editions resolve to the wrong thing. Strict matching fixed that but left popular titles unfound, so we added the "clearly most popular" rule with a test for the ambiguous case.
  • "Not in the top 50" is not "no". Our first aesthetic check only looked at the 50 strongest tags, so a missing look proved nothing. Now each look is looked up as a Qloo style tag and ranked against the full list.
  • Thin data. Niche comparables often return little signal; the verdict becomes "Not established", and an idea with no affinity scores is held instead of ranked.
  • Ties that looked like orders. Early versions said "make #1 first" when two ideas were nearly equal. Ties are now held and the screen says what would settle them.
  • Open access without a runaway bill. Judges need free access, so a daily LLM budget protects cost; past it, custom ideas still run without the LLM and the screen says so.

Accomplishments that we're proud of

  • Every Qloo value on screen traces to the call, entity, field and value behind it, and scores are computed from those values, not written by the model.
  • A guessed look is judged by its actual rank among all the style tags Qloo ties to the audience, not by whether a model thinks it fits.
  • An agent can run the whole check as one MCP tool call.
  • Three real decisions frozen before they were run, including one where we did not follow Pretaste's own ranking.
  • We report an evaluation we did not win (below).

What we learned

We kept losing to the LLM alone, and we report it. After each round of fixes we sealed a new set of six inputs the code had never seen and ran it once: same model on both sides, a blind Claude judge, three of the inputs carrying a planted wrong belief from the maker (for example "motorcycle ride logs sell mostly to girls under 20").

Sealed set Planted beliefs caught (of 3) Same top idea over 3 runs Blind judge preferred
1 2.0 vs 3.0 0.33 vs 0.67 0 of 6
2 2.5 vs 3.0 0.67 vs 0.67 0 of 6
3 0.5 vs 3.0 0.33 vs 0.67 1 of 6
4 2.5 vs 3.0 1.00 vs 1.00 1 of 6
5 2.5 vs 3.0 0.67 vs 1.00 1 of 6
6 2.5 vs 3.0 0.67 vs 0.67 0 of 6
7 1.5 vs 3.0 0.67 vs 0.67 1 of 6
8 (popular entertainment only) 3.0 vs 3.0 0.67 vs 1.00 1 of 6
9 (same scope, current version) 3.0 vs 3.0 0.67 vs 1.00 3 of 6

Pretaste first, the LLM alone second. Separately from winning, 0.75 to 0.92 of Pretaste's claims in each run were tied to a recorded Qloo value; the LLM alone has no data to cite, so its share is 0 by construction. The judge's reasons, in short:

  • Sets 1–5: when the comparables miss the subject (films and anime for a bonsai journal), Qloo faithfully describes the wrong fans and Pretaste carries them into the listing. Where they fit (motorcycle and hunting logs), the judge preferred Pretaste once for rejecting the planted belief from recorded values.
  • Set 6 (run four times, 0 of 6 each): Pretaste holds when the data cannot decide; the LLM alone gives a clear call with practical reasons and a fuller listing.
  • Set 7: the agent now picks a same-kind bestseller for books people use, but Qloo search often did not find it. The one win was a surf journal, where the "women over 65" belief was rejected from a recorded value.
  • Set 8 (sealed to test our own question: does it win where Qloo's coverage is strongest?): products bought by fans of horror films, anime, true crime podcasts, K-dramas, classic rock and retro games. Both sides caught all three planted beliefs, and Pretaste backed each correction with a recorded value (for anime, the 55-and-older band at −0.34). It still won only 1 of 6: when scores tied it held, and it missed practical points the LLM alone raised, such as KDP not printing loose card decks and drama quotes being copyrighted.
  • Set 9 (sealed before the run, with a pass mark fixed in advance: 4 of 6): the practical read gets its own model call and the result leads with its start pick when the data ties. Wins rose from 1 to 3 of 6. That is below our own mark, but no measure got worse, so this is the version now live. The judge's main reason in the losses: the start pick named one idea while the data checks and the listing were built around the other.

Data access alone does not beat a strong LLM. The comparables decide whether the buyer picture improves, and the decision still needs the practical checks a strong LLM already makes, which is why both are first in What's next.

Limits, in one place: Qloo describes the fans of the comparables the agent picked, not the buyers of this product. The agent picks comparables fresh on each run, so the same ideas can come out in a different order; that is why gaps under 8 points are held and age and gender are re-checked with each comparable left out. Taste affinity is not purchase intent; the research priority is a transparent formula, not a sales forecast; life stage is never checked; a region guess can only be supported (one guessed state in Qloo's top three), never challenged, because only the top states are recorded; US data doesn't speak for other markets. Sales impact is not measured yet.

What's next

  • Keep the LLM's practical checks (can this marketplace make it, rights risks, substitutes) next to the Qloo verdicts, so a tie on taste still ends in a clear call, and run the deep checks and the listing on the idea that call picks (the live version does the first half; it reached 3 of 6 in Set 9).
  • Keep the comparables for an idea fixed between runs, so a re-run can be compared with the last one.
  • Run each idea with a second set of comparables and keep only the verdicts that hold on both.
  • Our own second round at the 30-day checks (Nov 17 and 18), and outside sellers whose products are planned by AI and sold to US buyers.

Try it

Built With

Share this project:

Updates

Submission history