AffinityQA

• 

  Verified preference checks — raw, delivered and Qloo baseline

Inspiration

A movie list can pass every format check and still belong to yesterday's profile. A stale cache or a mismatched tool response can silently survive a change in taste. We built AffinityQA to make that failure visible: which profile reached the decision, what broke, what changed, and what evidence supports recovery?

What it does

Built for teams engineering personalized agents, the hosted workspace gives judges two complementary paths:

• Recorded repair: select a controlled incident and follow Detect, Diagnose, Repair and Verify. Compare the requested interest, consumed Qloo context, faulty output, repaired output and healthy reference. These genuine saved experiments can be replayed without new provider calls.
• New individual case: open a private browser session, declare two musical interests and optional film favorites or exclusions, review the plan, and explicitly confirm the artist identities returned by Qloo. Inspect five new films per profile, the original model ranking, the Qloo baseline and a downloadable verification receipt. This short preference route is separate from the controlled-fault campaign.

Reviewers can complete another case in the same session and restore their saved cases after a reload. They need no account, Docker installation, local model or API key. Fresh, cached and restored work are labelled separately. Musical interests supply context for film recommendations; song recommendations and music playback are not current functions.

How Qloo is used

Qloo supplies real artist-identity candidates and cross-domain movie-affinity context for explicit musical interests. The agent consumes that context to rank a fixed twenty-film catalog. Known favorites guide both Qloo and the model; favorites and exclusions stay outside the delivered five.

The evaluator follows identity across the requested interest, transmitted model input, consumed tool context and output. Three deliberately injected incidents break that chain: a cache key omitting the profile, a stale request profile and another profile's Qloo response. Supported repairs change the adapter's session routing or cache configuration, then verification checks recovery against healthy behavior and legitimate reuse. It does not claim arbitrary source-code repair.

How we built it

Next.js and TypeScript provide the review workspace; Python and FastAPI run the capture, diagnostics, repairs and file verifiers. Persistent storage retains plans, confirmed identities, original observations, failed attempts and selected verification receipts.

The original controlled experiment used a local model. The hosted operator uses Groq Free with GPT-OSS 20B. A durable provider queue reserves conservative request and token capacity before dispatch. Accepted work can be restored without resending it; exact Qloo context can be reused privately with explicit provenance. There is no lifetime application count cap, while shared provider quotas and storage safeguards still apply. Free inference is finite, and waiting estimates are not promises.

Challenges we ran into

A real artist search contained mixed musical and nonmusical duplicate IDs. Discovery v3 validates raw rows before projecting selectable musical identities, collapses only identical musical candidates and rejects conflicting identities. A later live trial exposed a private-cache validator still applying the stricter ranking contract to those searches. We aligned cache semantics with the pinned versioned identity contract and repaired the reader for historical partial search receipts. The failed trials, original responses and consumed work remain intact.

The final engineering review also hardened restart handling for terminal jobs with unsent reservations and recovery after an ambiguous browser response. Original cases and consumed requests remain preserved.

We also separated application access from provider capacity. Repeatable sessions do not create unlimited model quota. The queue reports waiting, preserves accepted work and exposes fresh versus reused context rather than silently retrying sent failures or switching to a paid provider.

Accomplishments and verified results

Earlier on 15 October, runtime 32406c2 completed two consecutive Sade / Nine Inch Nails cases: first 4 fresh Qloo requests and 2 Groq decisions, then 4 cached Qloo responses and 2 fresh Groq decisions. Both native downloads and restoration of the first case passed. These historical trials remain separate from the new Portishead / Bad Bunny case.

What we learned

Profile integrity, backend comparability and cultural quality answer different questions. A pipeline can pass integration checks while remote backend fingerprints vary; causal attribution for that separate 39-decision remote campaign therefore remains INCONCLUSIVE. A coherent recommendation list also does not establish that a person would enjoy the films.

Both latest top-five deliveries equalled the same-input Qloo baseline: this demonstrates integration and preference constraints, not improvement over Qloo. Cultural quality remains NOT_VALIDATED, verified human participants 0, and historical quality failures retain their scores and thresholds. Role-play inputs are synthetic engineering inputs. The tested-recommender release gate remains BLOCKED; that is an evaluator finding, separate from the diagnostic web application's availability.

What comes next

Independent human preference evaluation and broader controlled fault coverage need their own evidence. We will keep the hosted review freely accessible during judging, with preserved results and a zero-new-call recorded path when shared inference capacity is unavailable. Groq stays free; no unlimited throughput or future uptime is claimed. Raw provider captures and credentials remain private; original code is MIT licensed and third-party data retain their terms.

Built With

• fastapi
• groq
• next.js
• python
• qloo
• react
• render
• typescript