Inspiration

As humans, we are eager to find the people who share our beliefs, passions, interests and goals. Whether it's a friend, an investor or a lover, you need to see eye to eye. Today, building these relationships depends on actually meeting someone who has most, if not all, of those things in common with us. Somehow, every day we walk past these people, unaware of the shocking similarities we share. Think about all of the amazing relationships that never happen because we don't stop to talk to these people, not because we don't care, but because we don't know how similar we are.

VR, AR and AI can bring a person's character into the physical space in front of you, so you know who they are and are even pointed towards the people you want to be around. If the guy sitting across from you shares that he grew up in the same hometown, is also into XR development, and climbed Mount Rainier the same year as you, you're 1000x more likely to introduce yourself, because you know he is choosing to show this in search of like-minded people.

So many people have their digital personas spread thin across different apps: Instagram, LinkedIn, X, Snapchat, Substack, their personal websites, Letterboxd, Apple Music and everything in between. When you meet someone, you usually walk away with one handle and miss the rest of who they are. Avarri brings it all together in one place, right in front of you.

What it does

Avarri gets to know you as you connect your digital persona, adding your personal, social and professional accounts in one place. When you first open Avarri, she asks you a series of questions to get to know you and the kind of people you would vibe with best. As she learns more about you, she builds a fact sheet that is cross-referenced with other nearby users to find what you have in common.

When someone is nearby, their profile picture, their name and what you have in common pop up in Avarri's passthrough environment. If you're watching a movie, talking to your agents or playing a game in your headset, you get a notification that someone nearby shares your interests instead. When you pinch their profile, it opens to reveal the information they chose to share.

The main Avarri card includes a title, hometown, university, biography, a highlights section where you can share what you're proud of, and an "Ask me about" section, so anyone who wants to meet you has an easy opener that you actually want to talk about. At the bottom of the card is a Wave button that tells that person you waved, which signals interest in starting a conversation and gives them the option to wave back. There is also a Request contact button, so at the end of a conversation you can stay in touch: once they accept, they're saved in your people with their WhatsApp one pinch away.

In the middle of the card is My World. My World opens a panorama as a fully immersive environment and plays a song they chose, which gives you the amazing feeling of stepping into that person's shoes. It's my favorite feature of Avarri.

On the left side is a list of icons for social profiles, websites and connected apps the person wants to share: their personal website, X, Instagram, LinkedIn, favorite YouTube videos, recently played Apple Music, Substack and so much more. Here you can dive into someone's digital footprint and learn about all the different parts of their life, so you're more informed from the get-go.

On the right side of the card lives a live feed connected to the person's Threads or X account, if they choose to link it. These apps are where people post what they're actually thinking about, so as someone walks by you can see their most recent comments, ideas and thoughts.

How we built it

I built Avarri in Unity on a Quest 3S, using Meta's XR SDK (v207) with passthrough on the whole time. Everything is controlled with your hands: Meta's Interaction SDK gives me the hand rays, and I built my own gestures on top (point and pinch to select, pinch and drag to move, two hands to resize, double pinch to close). Meta's Depth API lets Avarri hide behind your real furniture and lets things sit on real surfaces.

Avarri herself is a figure made of about 3,000 points of golden light, drawn with a custom shader. She talks through an ElevenLabs voice agent, and while you talk she fills in your card live using tools I gave her. Claude writes the lines on your card, finds the "you both..." things two people have in common, and describes the photos you share. Headsets find each other over Bluetooth with a native Android plugin that both broadcasts and scans, plus a background service, so a match can reach you even when Avarri isn't open. Behind it all is my own Node server on a small cloud machine that stores cards, photos, videos and panoramas, and pulls in live posts, music, films and videos from the accounts people connect. To get photos in, the headset camera reads a code on your phone screen and the photos float out of your phone into the room.

Challenges we ran into

  • Pointing with your hands. The first hand rays I used were 30 to 50 degrees off from where I was actually pointing, so small buttons were impossible to hit. I rebuilt how aiming works and made buttons react when you let go of a pinch instead of when you start one, so scrolling and swiping stop pressing things by accident.
  • Keeping the AI honest. Early on Avarri would make things up, like adding a state to someone's hometown that they never said. I rewrote her rules so she only writes what you actually tell her, and you approve every line.
  • Sound and video on the headset. Getting a song or video to play on Quest took four different approaches before one worked reliably.
  • Teaching hands-only interaction without a wall of text. I went through several versions of the first five minutes before landing on a short animated hand lesson, a story, and Avarri walking you through a real card.
  • Building it solo against the clock. Design, the headset app, the server, the AI and the video were all one person.

Accomplishments that we're proud of

  • Two headsets finding each other over Bluetooth, even while one of them was in another app.
  • Avarri building a real card with you through conversation, line by line, in a few minutes.
  • My World. The first time I stood inside my own panorama with my song playing was one of the most beautiful moments of my life.
  • The whole app works with just your hands, from the first second to the last.
  • Privacy that's built in, not bolted on: everything is opt-in, you approve every line, and finding people uses Bluetooth, never face recognition.

What we learned

The card works best when it's written with you, not about you. People open up when an AI asks good questions and then lets them fix anything it gets wrong. I also learned that the most important moment in Avarri doesn't happen in the headset at all: it's when you look up and actually start talking to someone. Every design decision got better when I asked how it gets two people to that moment faster.

What's next for AVARRI

Avarri's future is as a social intelligence agent. Avarri can only learn so much about you from a first conversation, which means she will need to connect to your other agents, learn from your social media and stay in the loop about what's important to you and what you want to share with others. As Avarri gets to know you across different parts of your life, she will be able to make more precise, better-informed and more unique recommendations when matching people. I truly see Avarri becoming a part of every person's life, because it gives us access to the most unique thing in the universe: each other.

Built With

Share this project:

Updates

Submission history