Xtended Work Environment

• 

  Xtended Work Environment

• 

  Smartwatch

• 

  Laptop

• 

  HandTracking

• 

  ImageTracking

Inspiration

For more Consumers to adopt MR, they need to see that they don’t need to wait for a "revolutionary" killer app, but MR can help to make their daily, fundamental computing tasks easier.

Today we are surrounded by smart digital tools, SmartWatches, SmartPhones, and Desktop Computers, each with thousands of functions. However, physical screen size and number of displays are limited in the real world, and this limits our productivity directly.

We believe that MR devices could solve this problem by placing MR UIs into our physical world. Our MR application, the Xtended Work Environment, helps users to leap beyond the physical limitations. It can project any number of virtual desktop shortcuts, that users can interact with gestures naturally, around any device screen.

What it does

Magic Leap app module

1. The Magic Leap app recognizes AR markers, and places MR Buttons on it.
2. When user tap the MR Buttons using hand gestures, the app sends requests to PC app.

Desktop app module

1. The PC app launches HTTP server and waits for the request from the Magic Leap app.
2. When the PC app receives a request, the PC app runs specific script for the request.
3. At the moment, the script launches the specification desktop application associated with the icon touched.

How I built it

We used following functions/technologies:

• Image Tracking function on Magic Leap
• Hand Tracking function on Magic Leap
• Node.js to host the HTTP server on Mac

What's next

• Add more function to MR button.
• Optimize the hand and image tracking performance for the Magic Leap app.
• Expand the sizes and variety of recognizable AR markers. (Originally, we intended to use smart watch as a AR marker. However, it can be recognized, but the recognition is inconsistent because of the size).

-------------------------------- Note that the following description is for Round 1 --------------------------------

Inspiration

Display size and number of display are limited in real world. And it limits our productivity directly. We believe that MR devices could solve this problem by placing MR UIs to our world. However, we think that we need to place MR UIs to right place. We would like to propose two example for it.

Smart watch : By placing MR UIs tied to smart watch, you can manipulate MR object easily and intuitively.

Laptop PC : By placing MR UIs tied to laptop, you can improve productivity with 2D applications. And the MR UIs could be adjusted to your situation. For example, if you use PowerPoint in your laptop, the MR UIs could be optimized for it. So it could be "MR Touch bar".

Feasibility

As attached, Magic Leap One can track registered image. By using this function, we will place MR UIs.

Smart watch : We will create a watch face application for Wear OS device.

Laptop PC : We will use keyboard for the image tracking. And we will create a desktop application which monitors foreground application. This information will be used to optimize the MR UIs.

Experience

Holo Presenter Tool :

This is a HoloLens application for HoloHack hackathon by Microsoft.

Link to Devpost

CAV -Cripto Asset Visualizer with ARCore- :

This is an AR application for TechCrunch Disrupt SF 2017 hackathon. This app is picked up in TechCrunch website.

Link to the article

Link to Devpost

Built With

• android-wear
• c#
• electron
• javascript
• mac
• node.js
• powerpoint
• unity
• websocket