Inspiration

We’ve all been helped at some point in our lives - whether from instructional material or personally from experts.

We’ve all had to coordinate with others, and have seen the friction that results.

We each have a pretty good vantage point on how the future of the workplace will require more rapid training for coming challenges.

What it does

Streamlines remote worker coordination and training.

How we built it

  • First we identified the overarching problem (front line worker training and coordinating), and focused in on a data centre use case.
  • We then moved on to ideating solutions, and building a storyboard once we settled on one.
  • Identified what could constitute our MVP (and what our highest hopes would be)
  • Set up infrastructure and software configuration
  • Constructed our demo environment with Microsoft guides. Began integrating Azure and Power BI.
  • Set up networking to communicate between a computer (with Leap Motion) and a Hololens.
  • Using Leap Motion and Azure Kinect to share point cloud and hands
  • Leveraged our mentors to overcome some specific technical issues.
  • Consumed massive amounts of caffeine and sugar.
  • Test, test, test and iterate
  • High fived each other for a great accomplishment!

Challenges we ran into

  • Wifi Network Issues (dropped connections, latency and blocked ports on UDP)
  • Integrating multiple apps into a single experience
  • Identifying the MVP while still using AR capabilities
  • Hololens 2 had issues with: Battery life, Connectivity, Crashes (device would lock up and just shut off, requiring a reboot), User management and switching users was a frustrating experience and required many air taps to login to multiple accounts (Windows 10, Office 365, etc.)
  • Our team agrees that the Hololens issues we ran into caused delays and risked our ability to show a seamless solution (otherwise, the device is very capable and hand tracking in AR was exceptional!)

Accomplishments that we're proud of

  1. A comprehensive demonstration of multiple AR experiences which have only recently become a reality
  2. The team orchestrated the content, framework and an MVP within a short time - we built a startup in just a few days!
  3. Our solution includes new processes for training and remote collaboration featuring AR capabilities such as hand tracking, remote assistance and tracking progress within a complex business process

What we learned

  • Hololens 2 (and AR/VR in general) enables new experiences and possibilities which have never been done before!
  • Integrating different data capabilities will be key to the future of enterprise XR
  • Experiential on the job training is possible without having an expert physically present
  • Working with new tools for shared experiences: Vuforia, Photon, Holographic Remoting

What's next for AREdge

  • We will be publishing a whitepaper with Microsoft to document our journey in finding and creating a Mixed Reality experience
  • The team expects to continue its open-source contributions and solidify the concepts we created together
  • The feedback we are receiving is phenomenal - our shared passion with the community proves we have a marketable idea, thanks to the tremendous potential of Mixed Reality

Built With

  • .net-core
  • azure-functions
  • azure-kinect-dk-(boday-tracking)
  • azure-pipelines
  • dynamics-365-web-api
  • github
  • holographic-remoting-2.0.18-(nuget)
  • hololens-2
  • leap-motion-(hand-tracking)
  • mrtk-v2.2.0
  • office-365-graph-api
  • unity-2019.2
  • vuforia
Share this project:

Updates