AUR+A

Inspiration

I, Josephine, have been working in the lovetech industry for 3 years developing products for both dating and intimacy. Along the way I've learned a lot about what kinds of experiences and spaces spark connection and joy between people. And I've also learned that most dating apps and networking experiences don't spark much connection or joy for people :) People still come together, but it seems to be despite the technology not because of it! In this hackathon I wanted to take these insights and explore a new vision for how to use augmented reality and AI to make a better human connection experience!

What it does

The basic idea for the experience is that you can curate a profile that appears in AR as an "Aura" around you. You can use this to communicate basic information, playful prompts for connection, how you want to be approached, etc. We imagine it personalized with text, images, and in the future even more interactive AR components.

For a business event, you can indicate if you’re a founder trying to raise money or if you’re a VC. For a festival, you can use to meet people who are into the same music as you, just indicate it on your aura. If you want to date, you can say what you’re looking for, what are you into, how you would like to be approach.

Just like previous social networks provided a new homebase for people to connect and share in the virtual world, we imagine AUR+A as a new homebase for people in the augmented reality world, anchored to physical space yet providing all the exciting possibilities of the digital world.

How we built it

We wanted to prototype two things. First, we wanted to be able to experience having a AR video feed that gives you information live about the people you're interacting with! We think this is the near future for a lot of different kinds of devices, and we wanted to play with how we can make it something fun and engaging. To prototype this, we built a simple ios app using ARkit and amazon rekognition to identify faces live in the camera feed and supply information about the identified faces on the screen. We sourced photos and information from people's introductions on the hackathon slack feed to play with how it would feel to have this technology in a natural place like a hackathon or networking event!

We also wanted to prototype how generative AI technology can assist in this kind of playful connection. In our simple prototype, we just have chatGPT write an icebreaker joke based on the profile of the user and the person they are engaging with. However, we also envision something like Twelvelab's vision understanding technology providing a much richer way to generate fun interactions based on what's happening in the shared physical space.

Challenges we ran into

Face Recognition -- We'd love to explore less privacy invasive technique for providing a similar experience, but face recognition was the simplest tool we could use to prototype the experience today.
AR Glasses - We think this experience will be ideal for the types of seamless AR glasses experiences coming to the market soon, but given the early stages of this tech we decided to prototype on iphone.

Accomplishments that we're proud of

Although it's a simple idea, the experience of walking around with an AI giving you realtime information about people around you and offering you playful suggestions about how to interact is really fun! And it really was a great icebreaker for us.

What we learned

We learned that people really found the experience to be fun, even with such a simple prototype! People were really engaged both using the tech and receiving the interactions.
ChatGPT dad jokes aren't too bad
We got some good feedback about what would make different kind of users feel safe and excited to use this type of technology. Knowing what types of people could approach them and having the experience contained in a timebox or a specific physical space was important for some people. Others wanted to have a more interactive experience in deciding how much of their AUR+A to reveal to strangers.
Originally we wanted to use some of the features of twelve but decided not to because we didn't have time to implement it as a real time experience. However, we realized we could explore a lot of what types of experiences deeper visual understanding could provide by playing with prompts and queries on pre-recorded videos, for example filtering connections by fashion or suggesting interesting things in the environment to chat about.

What's next for AUR+A

We think providing a platform and example of how people's physical presence can be enhanced in the AR space to promote connection, playfulness, and creativity is a really exciting and open space. We're particularly excited about use cases like networking events and dating. We'd also love to explore a more gamified version of the experience (think real life quests, etc.), where users ... also, we really think it'd be fun if there was a mode where you could see everyone naked (imagined by ai, of course ;)