CompCheck

• 

• 

• 

• 

• 

• 

• 

• 

Inspiration

Genre is a filing system, not an audience.

Every film and TV pitch deck leans on comps: two or three famous titles that supposedly prove the new project will find its audience. A sales agent prices international pre-sales off the comps in the deck, and a financier stress-tests them before page one is read. So a comp is not decoration. It is a claim, and it is actually two claims: that the new project resembles the comp in tone, and that it connects to the comp's audience. A comp can be right on one and wrong on the other, and pitch-room logic cannot tell which, because sounding plausible is all a pitch room, or a language model, can measure.

I hit this building the deck for a mythological-horror feature in development: my comps looked right, sounded right, and nobody could tell me whether they were true. CompCheck is the check I needed. It tests both claims, separately, against Qloo's taste graph, and shows the evidence behind every verdict.

One scope note, up front: CompCheck does not price a film, forecast revenue, or measure how many viewers two films actually share. Pricing is the motivation. The product interrogates the evidence behind the comps a price gets argued from.

What it does

CompCheck is a closed-loop verification agent for pitch comps. It tests each comp on two independent axes, live against the Qloo API, assigns one of four verdicts, and then uses those verdicts to repair the deck. Its agency is deliberate and auditable: a state machine of decisions (resolve, verify, prune, repair) that replaces stochastic plausibility with rules a judge can inspect, not a chatbot asked to have an opinion.

Try it in 60 seconds. The demo is prefilled with a folk-horror pitch (reference: Tumbbad; comps: The Witch, RRR). Run it in Qloo Grounded mode and read the verdict cards; then add Bulbbul as a comp and run again.

Why a reference comp? An unmade project does not exist in Qloo's catalog, so there is nothing to score it against directly. What a deck actually asserts is that its comp cluster coheres: that the anchor title and the supporting titles point at the same tone and the same audience. CompCheck tests exactly that claim, internal comp coherence, against an explicit reference comp. You can designate the reference, or let Auto mode select it from your logline and genre (keyword matches against each comp's Qloo tag profile, with the scores shown in the trace), so verdicts never depend on the order titles were typed.

Test 1: the tone axis. Each comp resolves to a real Qloo entity via GET /search. The agent builds its taste profile from Qloo's curated tags (subgenre, theme, plot, style), merges duplicate tag names across urn types, and excludes broad genre labels from the score: Drama and Horror measure filing, not tone. Tone overlap is the Jaccard similarity of the two films' filtered tone-tag sets, against a calibrated threshold of 10%. It is a transparent similarity heuristic, not a measure of artistic equivalence.

Test 2: the audience axis. One batched GET /v2/insights call scores every comp, plus three unrelated control films (Dangal, Paddington 2, Top Gun: Maverick: a Hindi sports drama, a British family film, a US action sequel, chosen as deliberate non-kin across industries), for Qloo affinity toward the reference comp's audience. A comp passes only if it beats the strongest control. That rule is the point: raw affinity flatters any famous film, so popularity alone is not evidence; specificity is. Two qualifications, stated plainly. First, Qloo affinity is a modelled taste relationship, not measured shared viewership, box office, or demand. Second, beating the strongest of three controls is a conservative, project-defined screening rule, not a statistical test. A pass means the comp passes under this rule.

Four verdicts, defined by the two tests. TRUE: tone passes and affinity beats the strongest control. TONE: tone passes, affinity does not. MARKET: affinity beats the strongest control, tone does not pass. FALSE: neither test passes. The labels describe outcomes of these rules, nothing more.

The measured demo deck (reference: Tumbbad; control baseline 81.1):

Comp	Tone (threshold 10%)	Shared tone tags	Affinity (baseline 81.1)	Verdict
The Witch (2015)	16.7% pass	Folk horror, Dark, Suspenseful, Atmospheric, Slow burn	77.1 fail	TONE
RRR (2022)	5.2% fail	Atmospheric, Character driven, Cinematic	86.5 pass	MARKET
Bulbbul (2020)	14.5% pass	Folk horror, Folklore, Slow burn, Moody	92.0 pass	TRUE

Read the table as the trap it exposes. Pairing The Witch with RRR around Tumbbad mixes what looks like a Western arthouse tone comp with a domestic mass-audience comp; each validates half the pitch and contradicts the other half. Bulbbul, the title the deck did not have, passes both tests. That reading is an interpretation of Qloo's affinity signals, offered as one; the scores themselves are the measurement.

Verify, prune, repair. A comp that verdicts FALSE is pruned from the signal before alternatives are computed, and the repair query is seeded with the validated cluster only (the reference plus any TRUE comps). Every alternative suggested is then held to the same two-axis test. CompCheck never recommends a title it itself fails.

Honest failure states. A title that does not resolve is UNRESOLVED, with the nearest resolvable alternative proposed rather than silently substituted. A failed API call is an ERROR state, never a verdict and never cached as "not in the catalog". When demographic data carries no signal, the UI says unavailable instead of guessing.

The Plausibility Baseline. The toggle's other mode is a scripted illustration of genre-logic reasoning, labelled as such in the UI: it accepts every comp the way a pitch room does, and it exists so the contrast is honest. No model runs in the baseline, and the UI says so. Every live claim in this submission is scoped to Qloo Grounded mode.

Cross-domain, mapped to decisions. For the validated audience, CompCheck shows the top artists (soundtrack and temp-score direction), podcasts (press-tour and trailer promotion channels) and brands (co-marketing and festival partners), plus the reference audience's demographic core, the buyer quadrant a sales agent actually prices against. These are exploratory directions from the same taste signals, labelled as such in the UI.

How we built it

CompCheck is a zero-build static web app (HTML, CSS, vanilla JavaScript) hosted on GitHub Pages. Glass-box auditability is the design: every Qloo request, every intermediate score and every agent decision is rendered in the UI as a trace, so a judge can audit the reasoning, not just the answer.

Credentials are handled the way they should be: all API calls route through a small proxy deployed on Vercel, which injects the API key from a server-side environment variable and allowlists the two endpoint families the app uses. The key is not in the repo, its git history, or the client, and rotating it needs no client change. (An earlier Cloudflare Worker variant of the same proxy is kept in the repo's proxy/ folder for reference.)

How Qloo is used:

• GET /search: resolve comp titles to taste-graph entities under strict exact-name matching, and harvest the taste-tag profiles. Disambiguation is a visible agent decision: three films are called The Witch, and the agent picks the 2015 film and states its reason (a year typed in the input overrides it).
• GET /v2/insights with signal.interests.entities + filter.results.entities: the audience axis, comps and controls scored in one batch.
• GET /v2/insights (movies, validated-cluster signal, release-year filtered): the repair pool for alternative comps, each re-scored on both axes before it can be suggested.
• GET /v2/insights (artist / podcast / brand, popularity-capped): cross-domain affinities per category.
• GET /v2/insights (demographics): the reference audience's age band and gender lean, shown only when the data carries a signal.

We tested /v2/analysis/compare on the hackathon host: it exists, takes two entity signals, and returns a per-tag comparison. The verdicts are computed from /search profiles and /v2/insights affinities instead, for an architectural reason: the tone test needs pre-score tag filtering (stripping broad filing labels and deduplicating across urn types) and the audience test needs batched control-group benchmarking, and both need every number on screen to trace to raw inputs the UI displays.

Verification is part of the build, not an afterthought. tests/run-tests.mjs runs the shipped engine functions against the live API: a 20-title resolution matrix where every title is verified to resolve to the correct entity (name and release year, Western and Indian titles alike, Tumbbad the anchor), the disambiguation and honesty paths, the calibration bands, and the demo deck computed end to end. 59 checks, all passing. Because the suite exercises a live service, read that as verification of the shipped engine against the API at test time.

Challenges we ran into

The first version measured the wrong thing, and called it the right thing. It scored comps with a single number, tag overlap, and described it as audience overlap. Qloo's own affinity data reversed the headline: the comp with the highest tag overlap had the weakest measured audience affinity, and a comp the tool rejected had one of the strongest. Rebuilding that into two axes, with control films as the audience yardstick, is the real project. The challenges section of most submissions is a list of obstacles overcome; ours starts with a measurement error caught by the sponsor's own data, because that is what actually happened. Transparent rules do not become valid just because they are transparent; the measurement has to support the claim.

Entity disambiguation was the second real problem. "The Witch" is three different films in the catalog, and a comp validator that silently grades the wrong film is worse than one that grades nothing. Resolution is therefore strict: exact-name matching, prominence with a stated reason when names collide, a year override in the input, and an honest UNRESOLVED (with a proposed alternative) when nothing matches. A nonsense title returns UNRESOLVED in the test suite; a simulated API outage returns an error, never a fake absence.

Calibrating the tone threshold was the third. Tag Jaccard over Qloo's granular taxonomy is naturally sparse: a pair's combined tone profile runs to 50-60 tags, so even genuine tone-kin score in the teens. We measured 13 pairs on the live API (7 known tone-kin, 11.5% to 27.1%; 6 known non-kin, 3.8% to 7.0%) and set the threshold at 10%, in the empty band between them. The full set is published in the repo, with its composition stated: two of its pairs also appear in the demo deck, and the third demo pair (Tumbbad x The Witch, 16.7%) was not part of calibration and lands in the kin band on its own. The threshold is a transparent, project-defined decision rule, not an industry standard, and the submission says so.

Accomplishments that we're proud of

A complete, live agent whose headline claims a judge can reproduce in one click: run the prefilled pitch, read two verdicts that disagree with each other for opposite reasons, add Bulbbul, watch a TRUE comp land. Delete the Qloo calls and CompCheck collapses to its scripted baseline, which is precisely the failure mode the baseline exists to demonstrate. That is the hackathon's test, answered head-on: without Qloo, there is no product.

Coverage we proved, not assumed: the resolution matrix passes on Indian titles (Tumbbad, Bulbbul, Stree, Kantara, Andhadhun) as precisely as on Hollywood ones. That is a verified claim about entity resolution and tag coverage; we make no equivalent claim that scoring behaves identically across markets, and broader validation is listed under What's Next. In a field built on US pop culture, verified non-Hollywood coverage is still a first-class asset, and it is the home turf of the audience this tool serves.

And the honest baseline. It would have been easy to leave a rigged demo mode ambiguous. Instead the baseline is labelled a scripted illustration in the UI and in this story, because a tool whose pitch is "stop accepting unverified claims" has to start with its own.

What we learned

A comp is two claims, not one: a tone claim and an audience claim, failing independently. Collapsing them into a single similarity score hides exactly the risk a buyer needs to see.

LLM plausibility is not evidence; a taste graph turns a pitch-room argument into a measurement. But a measurement does not automatically prove the claim you want to make: we had to learn that distinction on our own first version.

And controls are what make a measurement a measurement. Without Dangal in the batch, an affinity of 77.1 sounds like a pass.

What's next

Logline-first mode (suggest comps for a pitch that has none, ranked by measured two-axis scores), territory market packs with control baselines swapped by target sales territory, broader calibration with stronger control selection and validation against independent assessments of comp quality, and a one-page producer brief export for sales agents and festival programmers.

Who pays is, at this stage, a hypothesis, stated as one: writers use CompCheck free, because writers build the decks; production companies, sales agents and development teams are the plausible paying users, because for them a false comp is a mispriced film, a wasted market trip, a slate built on a filing system. And the wedge widens: comps are how every creative industry sells the future, in publishing query letters, music release plans, branded content and podcast pitches. Somebody should sell the receipts.

Built With

• css3
• github
• html5
• javascript
• qloo-api
• taste-ai
• vercel