Experience Machine - Apple Vision Pro

A platform for 3D experiences - built for Apple Vision Pro

Comment

Inspiration

Watching the birth of the universe, editing the laws of physics, bending light with your hands - these were experiences previously reserved to dreams.

Now, it's all possible with the Experience Machine.

What it does

Experience Machine is a content platform similar to Youtube or TikTok. However, instead of videos, Experience Machine hosts novel, 3D experiences. There is an infinite amount of captivating experiences possible in a 3D, digital world. Individuals will be able to create and upload their crafted experiences in addition to the experiences developed in-house at Experience Machine.

The functioning experience I built for this version is a solar system environment with the planets orbiting around the user. The user can drag a slide bar to speed through time to view the state of our solar system across history and posterity.

How I built it

I build this product using Apple's in-house stack for developing on the Vision Pro, this includes SwiftUI, RealityKit, and Reality Composer Pro.

Challenges I ran into

  • Very little documentation on developing on Vision Pro given its recency
  • Loading in external 3D objects wasn't working, I had to custom make every 3D object in Reality Composer Pro
  • Couldn't figure out how to reconcile 2D styling in a 3D space. I had to abandon adding buttons because of this and opted for a slider
  • Couldn't figure out how to smoothly speed up the orbits as user manipulates time. The planets' orbit positions skip to their updated coordinates rather than smoothly zipping through the sky to get there
  • Much more

Accomplishments that I'm proud of

  • This is my first swift project
  • This is my first AR/VR project
  • I'm now one of the first developers in the world to develop on the Vision Pro
  • Built a repeatable method to display and execute unique experiences

What I learned

  • 3D objects need detail on their surface material to be pretty
  • I should have had a more robust plan of features to save time deliberating and backtracking

What's next for Experience Machine

High Priority

  • Understand what sort of experiences I would want to have in a 3D digital world
  • Investigate opportunities for spatial audio
  • Update the material/skin of the 3D objects to increase aesthetic appeal or figure out how to import external objects

Low Priority

  • Get user feedback on first batch of experiences (understand what people want)

Built With

  • realitycomposerpro
  • realitykit
  • swiftui
Share this project:

Updates