Training is large expense in enterprise. Particularly, in situation where inexperience operator needs help from an expert who is not next to him, the best way for instruction is through skype or phone call. However, it still can be ineffective, and it would be better if the experienced expert is right next to the operator to teach him side by side.
What it does
In our current hack, we can draw annotation in different colors in 3D. This annotation should be transmitted in real time to operator's view if we have networking work properly, and the annotations should appear next to operator's real object as it appear to annotations to the hologram that expert sees.
How I built it
Unity, Kinect Fusion, Python, Flask
Challenges I ran into
Networking is very hard to set up on hololens
Accomplishments that I'm proud of
We came up with a novel way to do hand-tracking with the Hololens, greatly improving the interactions possible with the device.
What I learned
We also ran VR demos for almost 8 hours, and led a workshop. It's hard to find time to get work done on the hack while juggling these other (really fun) events!
What's next for PenGram AR
Getting remote collaboration to work over our server. It's 99% done now, just needs a couple finishing touches.