TasteProof

Cultural QA for AI agents: audit whether Qloo-grounded taste evidence actually improves a recommendation.

Comment

The problem

AI agents can make recommendations that sound culturally plausible while being weakly grounded. For teams building recommendation, planning, creator, brand, travel, or entertainment agents, the real engineering question is not “can the model explain this nicely?” It is: does the recommendation hold up when tested against cultural evidence?

What TasteProof is

TasteProof is cultural QA / regression testing infrastructure for AI agents. It evaluates an agent’s proposed recommendation against explicit, user-supplied cultural anchors using Qloo’s taste graph.

It is deliberately not another recommendation chatbot. The core unit of value is a reproducible cultural test.

Judge path

  1. Enter a scenario being evaluated.
  2. Enter 1–3 explicit named cultural anchors.
  3. Choose the recommendation entity type.
  4. Enter the AI agent’s proposal being audited.
  5. Run the live Qloo audit.
  6. Read the Taste Passport: cultural-fit classification, proposal rank, better-grounded alternative when one exists, provenance, limitations, run ID, and SHA-256 digest.
  7. Expand the evidence record to inspect the Qloo workflow results.

The separate Illustrative audit is synthetic and clearly labeled. It demonstrates the product experience only and never counts as Qloo evidence.

Why Qloo is structurally necessary

Without Qloo, TasteProof can collect an agent proposal, but it cannot perform its central cultural-grounding test.

The live evaluator uses two supported Qloo hackathon workflows:

1. recommend — creates a Qloo-grounded comparison set from the same explicit taste anchors.

2. rank — places the audited proposal and Qloo-grounded alternatives into one shared comparison context using those same anchors.

That shared comparison matters: TasteProof does not compare unrelated scores from separate calls or invent a proprietary “cultural score.” It asks a concrete question: where does the agent’s proposal rank when tested beside grounded alternatives under the same evidence conditions?

If the proposal ranks first, TasteProof reports it as supported within that run. If a grounded alternative ranks higher, the proposal is flagged as weak relative to the tested set and the better-grounded alternative is surfaced. If Qloo cannot resolve enough evidence, TasteProof reports uncertainty instead of manufacturing a conclusion.

Taste Passport

A successful live run exposes:

  • scenario
  • explicit cultural anchors
  • agent proposal
  • Qloo entity type
  • cultural-fit classification
  • proposal rank within the tested comparison set
  • better-grounded alternative, if present
  • Qloo recommend evidence
  • Qloo rank evidence
  • live provenance
  • execution boundaries
  • limitations
  • run ID
  • semantic SHA-256 evidence digest

The digest excludes volatile execution metadata so identical semantic evidence can be traced more consistently.

Reliability and failure behavior

TasteProof inherits reliability practices from prior agent-system experimentation:

  • server-side credential only
  • official @qloo/qloo-harness 0.1.26
  • canonical qloo exec workflows
  • maximum three explicit anchors per run
  • maximum six compared options per run
  • bounded workflow timeout
  • no hidden provider fallback
  • visible unresolved or ambiguous entities
  • deterministic input validation
  • fail-closed live path
  • separate fixture and live provenance

If the Qloo credential is missing or a live Qloo workflow fails, the production endpoint stops visibly. It does not silently substitute fixture data.

Technical implementation

  • Node.js 22+
  • Vercel serverless deployment
  • @qloo/qloo-harness 0.1.26
  • qloo exec recommend
  • qloo exec rank
  • server-side Qloo environment configuration
  • deterministic helper tests for validation, anchor deduplication, relative-rank classification, proposal-rank mapping, and semantic digest stability
  • SHA-256 run lineage
  • MIT license

Responsible-use boundary

TasteProof uses explicit cultural anchors. Qloo results describe aggregate taste patterns; TasteProof does not treat them as proof of an individual’s identity, sensitive traits, health, finances, politics, or future behavior. The result is a relative test against the options evaluated in that run, not a universal cultural truth or a high-impact eligibility decision.

Who it is for

The immediate audience is developers and product teams building culturally sensitive AI agents. The broader opportunity is a CI-style cultural evaluation layer: when a prompt, model, ranking policy, or product constraint changes, teams can rerun the same cultural scenarios and detect whether recommendation grounding improved, regressed, became ambiguous, or lost provenance.

What makes TasteProof different

Many cultural-intelligence products use Qloo to generate recommendations. TasteProof uses Qloo as the evidence layer for evaluating another agent’s recommendation.

That makes Qloo not decorative, but structurally necessary—and turns cultural grounding into something developers can inspect, reproduce, and regression-test.

Built With

Share this project:

Updates