Fieldnote discovers and scores artists for music sync placements using Qloo's cross-domain taste graph across 8 cultural domains

Inspiration

Music sync licensing is a $1.5 billion industry, and the discovery step is still shockingly manual. A music supervisor gets a brief like "indie coming-of-age drama, rooftop scene at sunset, 18-34 female skew," and what do they do? They open Spotify, scroll through playlists they already know, email contacts they already have, and pitch artists they have placed before. The actual question they need answered, "which artists have fans who are the same people watching this show?", is not something any playlist can tell them. That is an audience overlap question, and Qloo's taste graph is one of the few places where that data actually exists.

I wanted to see if I could turn Qloo's cross-domain cultural data into something a supervisor could hand to a client: a scored, explained recommendation, not just another list of similar-sounding names.

What it does

Fieldnote is a music sync placement agent. A supervisor fills out a brief the same way they would in real life: project type, mood, target audience, genre preferences, exclusions. Within about 15 to 20 seconds, Fieldnote resolves the project through Qloo's cultural graph, walks taste connections across 8 domains (Music, TV, Film, Podcasts, Books, Games, Brands, People), finds which artists live in the same cultural world as the project's audience, scores each one on three independent axes, and surfaces a ranked shortlist with transparent reasoning, Qloo taste tags, and inline Spotify playback so you can actually listen.

The output is not just a list of names. Each artist carries a composite score broken into three parts: cultural fit (how closely their Qloo taste profile overlaps with the project's audience), creative fit (would this artist's sound actually work in this specific scene and project type), and audience relevance (do this artist's real listeners match the demographics of the project). Every score traces back to Qloo data or the evaluation chain that produced it, so a supervisor can defend the recommendation in a client meeting instead of just saying "trust me."

Here is how the three axes work and what they weigh:

Cultural fit (50%) — Qloo resolves taste tags for both the project and the artist. Fieldnote compares them directly: how much overlap exists between what this artist represents culturally and what the project's audience gravitates toward. This is pure Qloo data, no LLM involved at all. Creative fit (30%) — An LLM (Claude Haiku) reads the scene description, the project type, and the artist's Qloo tags, then judges whether this artist's sound would actually work in that specific placement. It never sees the cultural or audience scores, so this signal stays independent. The evaluation weights 60% use-case fit and 40% mood fit, with per-project-type guidance baked into the prompt because a trailer needs very different music than a documentary, and the prompt knows that. Audience relevance (20%) — Qloo's demographics endpoint gives real age-bracket and gender-affinity distributions for each artist. Fieldnote compares that demographic shape against the project's audience profile to see how closely this artist's actual listeners match the people who would watch, play, or engage with the project.

The composite score ranks the shortlist. All three component scores, the full reasoning chain, and the Qloo taste tags are visible on every result card. Nothing is hidden behind a single number.

Two pipeline modes

Entity mode (6 steps): The supervisor names a project, say "Stranger Things." Qloo resolves it as a cultural entity, and Fieldnote pulls the full cross-domain taste graph across all 8 domains. That graph shows what the project's audience loves beyond just music: the podcasts they listen to, the games they play, the brands they follow. That cross-domain signal is what drives both artist discovery and cultural fit scoring, because an artist who shares taste connections across podcasts, books, and brands with a TV show's audience is a stronger match than one who just sounds similar.

Description mode (5 steps): No named project. The supervisor just describes a scene and a mood. Since there is no Qloo entity to anchor on, there is no taste graph to walk. So instead, the LLM infers a target audience from the scene description and project type, expressing it in Qloo's own demographic format (the same 6 age brackets and signed gender affinity that Qloo reports), so the audience scoring pipeline works the same way in both modes. Fieldnote then queries Qloo's music domain directly with mood and genre terms from the brief to find candidates.

Both modes run the same three-axis scoring and give the same structured output. The difference is what powers discovery: a full cross-domain taste graph in entity mode, direct music-domain queries in description mode.

How we built it

Next.js 16 (App Router), TypeScript, Tailwind CSS v4, shadcn/ui, deployed on Vercel. The pipeline runs entirely server-side through SSE streaming, so the supervisor watches each step complete in real time: entity resolution, domain queries, artist discovery, demographic pull, creative evaluation, final scoring, each with timing and status.

Qloo is the center of the pipeline, not a side call. Every run hits Qloo's search, similar, insight, and demographics endpoints across up to 8 domains. In entity mode, the agent resolves the project name through Qloo, retrieves cross-domain taste connections, finds candidate artists through taste similarity, and pulls demographic distributions for each one. Qloo is the only source of cultural and audience data. Nothing is cached from training data or hardcoded.

I kept the LLM deliberately narrow. Every LLM call goes through a single function (callProvider), scoped to one job: creative-fit evaluation (and audience inference in description mode). The LLM gets Qloo's taste tags as grounding context but never sees Qloo's numerical scores or demographic data. This separation matters a lot. Cultural fit and audience relevance, 70% of the composite, are entirely Qloo-derived. If I removed the LLM, those two axes would still work. If I removed Qloo, nothing would work.

Concurrency was tricky. Qloo's hackathon API allows 5 requests per second. A single entity-mode run can generate 40+ Qloo calls across 8 domains plus demographics for each candidate. I built a process-wide semaphore with 25 concurrent slots that gates throughput across multiple users without tripping rate limits. LLM calls share the same budget.

I added an in-memory response cache late. Repeated LLM prompts within the same server process now resolve instantly from a SHA-256 keyed cache. But I learned the hard way that you cannot cache blindly, because occasionally the model returns truncated JSON. If that broken response enters the cache, the retry mechanism hits the cache, gets the same broken response, and that artist permanently fails scoring until the server restarts. So I added a JSON validation guard: it runs JSON.parse on the response before storing it. If that fails, the cache is skipped and the retry gets a fresh API call.

Challenges we ran into

Claude Haiku rejects the temperature parameter. The Anthropic API returns a 400 when you set temperature for claude-haiku-5-5. Same for top_k. Both are documented as valid in the API reference, both fail at runtime. I found this through 400 errors on every LLM call after deploying, not from reading docs. The fix was simple: strip both parameters and accept the model's default sampling. But it cost me about two hours of debugging what looked like an auth issue before I read the response body closely enough to see the actual rejection reason.

Qloo's demographic data uses signed affinity values, not percentages. The age bracket values range from roughly -0.7 to +0.7, representing how much a given audience over-indexes or under-indexes relative to the general population. I initially treated them as raw percentages and got nonsensical audience scores. I had to build a comparison function that works on the shape of the distribution, which brackets over-index, rather than the absolute values. For description mode, where there is no Qloo entity to read demographics from, the LLM infers a synthetic demographic profile in the same signed format, so both modes feed the same scoring function without a translation layer.

Caching malformed LLM responses nearly broke scoring. The first version of the cache stored every non-empty LLM reply. When the model returned truncated JSON (which happens occasionally under token limits), that broken reply entered the cache. The retry sends the same prompt, hits the cache, gets the same broken reply, and that artist permanently fails creative scoring until the server restarts. A subtle bug that only shows up under load and never reproduces when you are looking for it. The fix was the JSON validation guard I mentioned above: parse before caching, skip on failure so the retry gets a fresh call.

Building description mode without a taste graph. Entity mode's real strength is Qloo's cross-domain taste graph. That graph does not exist when there is no named entity. Rather than forcing the supervisor to always name a project, I built a fallback path that queries Qloo's music domain directly using mood and genre terms from the brief, then infers a demographic profile from the scene description. The tradeoff is real: description mode has no cross-domain signal, so cultural fit scoring relies on direct music-domain similarity rather than a full taste graph. The scores are still meaningful, but they are narrower. I chose to show this in the UI rather than hide it.

Accomplishments that we're proud of

Qloo actually drives the result. The easiest path would have been to send the entire brief to an LLM and ask it to recommend artists. That would produce confident, plausible, and completely ungrounded answers. Instead, Qloo's taste graph powers cultural fit (50% of the composite), Qloo's demographics power audience relevance (20%), and the LLM is confined to creative-fit evaluation (30%) where it gets Qloo's tags as grounding. If you remove the LLM, the result degrades. If you remove Qloo, it breaks entirely. That dependency is intentional and I think it is the right architecture.

Per-project-type creative evaluation. A trailer and a documentary need completely different things from music. The creative-fit prompt carries project-type-specific guidance: a trailer supervisor needs music that can be cut to picture in 15-30 second bursts with strong dynamic range; a documentary supervisor needs music that sits underneath narration for long stretches without competing. This guidance reshapes the LLM's actual scoring criteria, not just its tone, and it is the reason the same artist can score 0.85 for a video game trailer and 0.45 for a podcast.

Real-time pipeline transparency. The supervisor watches each step complete with timing and status. A failed domain query shows as failed, not silently omitted. If you are a music supervisor deciding whether to trust a recommendation, you need to see what went into it. Opaque confidence is not useful in this industry, and I did not want to build another black box.

Cross-domain discovery, not music-to-music matching. In entity mode, Fieldnote queries all 8 Qloo domains and uses the cross-domain taste graph as the primary discovery and scoring signal. An artist who shares taste connections with a TV show's audience across podcasts, books, and brands is a stronger cultural match than one who just sounds similar. That cross-domain walk is what makes this a Qloo-native tool, not a recommendation wrapper with Qloo on the side.

What we learned

Qloo's taste graph is most valuable when it crosses domains. Music-to-music similarity from Qloo is useful, but the real power is the cross-domain data. When Qloo shows that a TV show's audience also gravitates toward certain podcasts, brands, and games, that maps the cultural world those viewers live in, and I can find artists who live in the same world. A recommendation engine that stays within music cannot see that signal. The cross-domain walk is the entire reason this project exists.

The LLM's job is judgment, not data. Every time I let the LLM generate factual claims, audience demographics, sync history, chart positions, it hallucinated. Constraining it to evaluative judgment ("given these Qloo tags and this scene, would this artist's sound work here?") produced consistently useful output. The lesson generalizes: LLMs and cultural knowledge APIs have complementary strengths, and mixing their roles makes both worse.

Scoring transparency matters more than scoring precision. Early versions showed only the composite score. Testers kept asking "why this artist?" The three-axis breakdown with visible weights, the reasoning chain, and the Qloo taste tags give a supervisor enough information to agree, disagree, or adjust. An opaque score, however accurate, is not something you can defend in a client meeting. That breakdown is what turned Fieldnote from a curiosity into something a supervisor might actually use.

What's next for Fieldnote

Licensing availability layer: connecting to sync licensing databases so the shortlist can be filtered by clearance status and budget range, bridging the gap between discovery and deal. Playlist-mode queries: accepting a reference playlist instead of a single project, giving the taste graph more input signal to work with.

A/B comparison: running the same brief against two project types side by side to show how project context reshapes recommendations. Deeper Qloo entity matching: using Qloo's insight endpoint for richer cross-domain profiles where available.

Geographic affinity scoring: incorporating geographic taste data when Qloo's API exposes it, adding a regional alignment axis.

Honest boundaries: Fieldnote is a hackathon prototype. The live app at "fieldnote-dusky.vercel.app" is fully functional and queries Qloo's API on every run, no mock data. It covers the discovery phase of sync placement, not licensing, budgets, or catalog rights. Description mode's audience inference is an LLM estimate expressed in Qloo's format for consistency, not a measurement. Cultural fit scores depend on Qloo's taste tag coverage per artist, and creative-fit evaluation depends on the LLM's training knowledge of each artist, both limitations I surface in the output rather than hide. There are no automated tests; verification has been manual through the live deployment.