Inspiration
Your AI agent is ready to launch. But is it ready for the next market? Imagine an online bookstore expanding into Chennai, Kuala Lumpur, and London. Customers ask the same question: “Recommend three fantasy novels for a birthday gift.”
The assistant can produce convincing answers everywhere. But when customers have not supplied their favorites, what guides its recommendations? Does it use meaningful regional evidence, or rely on familiar defaults? Now imagine a customer does share their interests. Can the business inspect whether those interests influenced the choices?
These questions inspired TasteAudit.
As data scientists, we wanted to make cultural relevance something teams can investigate. TasteAudit combines an evaluation workflow with Qloo’s cultural intelligence to examine recommendations across markets and individual interests.
What it does
TasteAudit is a cultural intelligence audit tool for AI recommendation agents. It provides two paths:
1. Regional Audit: What should change across markets?
Users enter a recommendation request, choose a domain, and add cities. Personal favorites are optional.
Gemini receives the request and the known market context. TasteAudit retrieves available regional taste evidence from Qloo, discovers alternatives, and produces a Qloo-assisted revision.
For domains without confirmed direct regional entity scoring, TasteAudit uses a derived approach:
City → regional taste tags → compatible candidates → scored recommendations
The interface distinguishes derived regional profiles from direct evidence. It does not present tag-conditioned book scores as directly measured preferences of every resident.
Users can inspect regional profile differences and compare:
- Original and revised recommendations.
- Mean observed affinity and score coverage.
- Added, removed, retained, and reordered items.
- The evidence supporting the revision.
- Unsupported signals and missing information.
2. Personalized Audit: Did the agent use the customer’s interests?
Users provide a request, practical requirements, and explicit favorites across cultural domains.
Qloo connects those interests to potential recommendations. TasteAudit evaluates the original selection alongside discovered alternatives and generates a grounded revision.
The interactive Taste Fingerprint visualizes available interest-level explanation scores. Users can inspect which supplied interests supported a recommendation, while missing attribution remains visible.
Our bookseller and concierge demos illustrate the workflow. The broader goal is to support teams building recommendation agents across retail, entertainment, hospitality, and cultural discovery.
How we built it
We began with Python notebooks to develop the evaluation workflow, inspect API responses, and test the comparison logic.
We then built the web application using Next.js, React, and TypeScript, deployed on Vercel.
Gemini generates original recommendations, extracts named entities, and writes revisions from supplied evidence.
Qloo provides entity resolution, cross-domain candidate discovery, supported regional taste analysis, affinity scores, and available explainability.
The pipeline separates each stage:
- Capture the complete customer or market brief.
- Generate the original Gemini answer.
- Resolve recommendations to canonical Qloo entities.
- Retrieve relevant regional or personal-interest evidence.
- Discover additional candidates.
- Evaluate originals and alternatives consistently.
- Generate a revision from eligible candidates.
- Compare selections and expose the evidence.
The original assistant receives the full known context. We do not withhold location or interests to create a weaker baseline.
Both recommendation sets are evaluated using Qloo. Affinity change therefore measures alignment with the evidence used in the audit.
The application runs session to session, requires no login, and keeps provider credentials server-side.
Challenges we ran into
Location is not one capability
Finding venues inside a city, retrieving regional taste tags, and scoring a book against a regional audience are different operations.
We had to check domain-specific support and make the applied method visible. When regional evidence is unavailable, the system must say so.
Reordering is not better selection
An early revision returned the same three books in a different order. Their unweighted mean affinity correctly stayed unchanged.
That exposed the need to inspect candidate discovery, distinguish replacements from rank changes, and avoid calling every rewrite an improvement.
Entity names can be ambiguous
Books have editions and adaptations. Places can share names. We needed canonical IDs and visible identity uncertainty rather than treating every plausible match as confirmed.
Explanations need evidence
A fluent explanation is easy to generate. A defensible attribution requires returned evidence.
We separated native Qloo explanation scores from application-computed sensitivity and avoided inventing contributions when explainability was unavailable.
Cultural fit does not verify practical facts
Affinity does not establish stock, price, opening hours, or accessibility. We kept practical verification separate from cultural evidence.
Accomplishments that we're proud of
We built two complementary audit paths: regional cold-start analysis and explicit-interest personalization.
We connected cultural discovery to an inspectable recommendation workflow, showing the candidates, selections, scores, coverage, and changes.
We also made measurement part of the interface. The regional comparison and Taste Fingerprint help users understand the evidence without requiring them to read raw API responses.
Most importantly, TasteAudit can report an unchanged result honestly. The purpose is to investigate recommendations, not manufacture a positive chart.
What we learned
Regional evidence can provide a useful starting point when individual preferences are unknown. It is still an aggregate signal, not a personal identity or a guarantee.
Explicit preferences, regional context, request relevance, and practical constraints each contribute different information.
We also learned that optimizing against Qloo and evaluating with Qloo measures internal alignment. Demonstrating better customer satisfaction requires independent human evaluation or observed outcomes.
Finally, evidence availability is part of the result. Missing scores, uncertain matches, and unsupported regional signals should remain visible.
What's next for TasteAudit
We want TasteAudit to become a reusable evaluation layer for teams building AI recommendation agents.
Our next steps include:
- Connecting external agents and importing their responses.
- Expanding domain and regional coverage where supported.
- Building fixed evaluation suites across markets and preference profiles.
- Comparing Qloo-assisted revision against Gemini self-refinement.
- Adding blinded human evaluation.
- Integrating verified catalogue, stock, price, and venue information.
- Measuring customer outcomes through controlled experiments.
- Tracking changes after model and prompt updates.
Across markets. Across interests. TasteAudit puts your AI’s taste to the test.
Built With
- gemini
- google-gemini
- next.js
- python
- qloo
- react
- typescript
- vercel
Log in or sign up for Devpost to join the conversation.