Inspiration

3OS is inspired by a treasure trove of ancient 90s XR HCI research papers whose techniques are all but forgotten in modern XR OS design, yet they are the key to unlocking high throughput for XR 2.0. See the github.com/scifiuiguy/3OS#readme Prior Art Section.

What it does

3OS uses a proprioceptive nested isotropic interaction model (instead of raycast/WIMP) to provide fast-twitch, voxel-based interactive access to the entire field in under 0.25 seconds, whereas raycast can only select nearest-in-field and it can't manipulate objects uniformly, yielding lethargic single-hand-to-single-object interactions.

How I built 3OS

I knew something was wrong with XR UX in 2019 when my team built the Spotify App on Magic Leap. The industry, the users, and even Academia were pushing to conform to raycast/WIMP in XR. I felt trapped by it, and that motivated me to discover SIGCHI, where I found some inspiring 90s studies that led me to build Lexting, the fastest spatial XR text entry system ever. I didn't fully understand the importance of isotropic design at the time, but Lexting was my first fully volumetric, isotropic interface. Since then, I've grown a passion for analyzing Fitts' Law-based XR papers so that the spatial computing industry can finally beat desktop computers at their own game by delivering the highest possible throughput for every task.

Challenges I ran into

Convincing colleagues and customers to break out of the box to discover the throughput gains we've been missing is the hardest part of building XR experiences, especially for enterprise. Consumer apps can afford to skimp on throughput in favor of experiential flavor and the comfort of legacy familiarity through similarities to touchscreen/desktop designs. Enterprise apps cannot afford to be so lax. Users will simply not switch to XR unless the task is faster in XR than on desktop.

Accomplishments that I'm proud of

In 3OS, you can cut/copy/move clusters of files in 3D space faster than the same operations on a desktop file explorer.

What I learned

Everyone in the post Steve Jobs/Jony Ive UX world assumes "intuitive interaction" is the Bible. If the user can't do the thing without explanation, then "you built it wrong." I discovered that's false for XR. While building Spotify at Magic Leap, we built a depth-based interaction that blew the mind of every tester once we showed it to them, but they never would have found it on their own. Their intuition was to use a Tinder-esque swipe-left/swipe-right gesture instead, a very mundane and inefficient version of the experience. Seven years later, legacy intuition is still the obsession of most XR UX designers. Old UX habits die hard and hold us back from XR 2.0, especially for enterprise.

What's next for 3OS

3OS needs the backing of a major player in the OpenXR Working Group to move forward. 3OS is the jelly to the peanut butter of OpenXR's new spatial container multitasking spec. A prototype of the on-prem edge-compute version (the 3OS Bay) is under early R&D, but it requires an EPYC 9004, a Pro 6000 Blackwell, and an office where a small team can dogfood the solution as it comes to life. A hybrid 2D/3D OS feels inevitable for professional 3D content creators (CAD engineers, game devs, VFX artists/programmers, or any software engineers), but only if it has the high throughput capability of 3OS. In 1995, Windows became ubiquitous as a consumer operating system. Windows NT emerged as a more robust version for enterprise. 3OS aims to become the Windows NT of XR, a faster OS for all professional 3D content power users.

Built With

  • cursor
  • openxr
  • unity
  • unitypipeline
Share this project:

Updates

Submission history