Fairs

Inspiration

Recommendation systems understand what content is about, but not how it feels to the brain. With Meta’s new TRIBE v2 model, we realized we could bridge that gap.

We wanted to build a system that can answer a completely new kind of query: “Find something that feels like this, but is about that.”

What it does

Fairs is a multimodal discovery engine that takes two inputs:

“Like this” → matches brain-response (neural) similarity
“About this” → matches semantic content

It returns media (video, audio, text, etc.) that combines both.

Example: Find something that feels like a Radiohead song, but is about space.

How we built it

Used TRIBE v2 to extract:
- content embeddings (what media is about)
- predicted brain activity (how it’s processed)
Precomputed embeddings for a media library
Built a FastAPI backend + FAISS retrieval system
Scored results using:
- content similarity
- neural similarity
Built a React frontend with an explainability view

Challenges we ran into

Working with a brand new research model
Computing embeddings for each brain map within 12 - 15 hours
Converting time-based multimodal outputs into clean embeddings
Balancing neural vs content similarity without noisy results
Making everything fast enough for a live demo

Accomplishments that we're proud of

Built a working brain-response-based recommendation system
Enabled cross-modal retrieval (e.g., song → video)
Created a query type no existing system supports
Made it explainable with brain-based reasoning

What we learned

Research models become powerful when paired with the right product idea
Multimodal retrieval is as much about UX as it is about modeling
Explainability is critical for anything involving “brain” concepts

What's next for Fairs

Scale the media library
Improve ranking + personalization
Expand brain-based explanations
Release to real users

Long term: We hope that this will encourage organizations and people to build search systems based on predicted cognition, not just content, enabling a new age of recommendation systems that are more accurate than ever before.