Be My Eyes XR

Inspiration

Geordie LaForge's vision tech, Daredevil's audio vision, RoboCop's AR vision, or Neo being able to see after losing his sight. Growing up, these technologies fascinated me, but after meeting Mo, a low-vision colleague, who used an iPad pro to see the world, I believe the technology to help people like him needs to accelerate.

With XR tech such as the Meta Quest, Apple Vision Pro, and the upcoming Meta Orion, I believe that that future can come soon.

For the immersive insiders XR Design challenge, I decided to take this opportunity to bring this concept to light and redesign what Be My Eyes could look like powered by XR tech.

For context: Be My Eyes is an award winning accessibility app where low-vision users can get help from volunteers using through video calls to be their eyes. Later, Be My Eyes won boost grants from Expo 2020 Dubai's Expo Live, an Apple Design award, and has collaborated with Open AI, and Meta Raybans.

What it does

In this video I showcase:

• A simplified onboarding that uses simple hand gestures rather than eye tracking, to interact with the app.
• A digital twin environment that recreates the world around the user, for better flexibility in showing information to those who need different types of visual assistance.
• An alternate High Contrast version to recognize shapes in an environment.
• A spatial audio ping that allows to locate objects in a space with sound.
• A visual AI that identifies objects direct in front of a user.
• Being able to call on a volunteer, who can help (A feature that Be My Eyes was built on)
• References of best practices I have taken inspiration from.

How we built it

• The concept was built using Figma, and built on Bezi. It was prototyped on to work on desktop, and the apple vision pro.

Challenges we ran into

• I wanted to design this in Shapes XR, but because of the holidays I didn't have much time to learn a new XR design software. I opted to stay with software I was familiar with - Bezi.
• Bezi, has some limitations for complex interaction design. I created the feature I required by using colliders as a timeline mechanism.
• Because of the number of animations, and colliders I had to make sure that the project was extremely optimized to reduce visual bugs. This took lots of time.
• Considering that the challenge happened to the end of year holidays, finding time to work on this was difficult. I decided to create a project that I knew I could do efficiently.

Accomplishments that we're proud of

• I am especially proud that I did this particular app. I am passionate about accessibility apps after meeting Mo who needs accessibility tech to navigate life. I hope that this video and concept will help convince designers at Meta, Google, and Apple to advocate and keep accessibility as part of their core deliverables.

What we learned

• I learned alot of accessibility design more than I had realized. I was surprized how far it's gotten, and all I had to do was reframe it within the context of XR. Because of all of these advancements, I believe what I have proposed is feasible in the next year.

What's next for Be My Eyes XR

• If I had the chance to take it further, I would seriously consider adding more features for different types of visual assistance needs. For example, my friend who needs visual assistance, has tunnel vision, Being able to compress the view and give a wider view within a specified visual area can greatly help with his mobility.

Built With

• bezi
• figma