Going into a project with the hopes of playing with cool hardware, we decided to combine two pieces of equipment that individual do great things, the kinect and Oculus. Individually, we could make some pretty neat hacks, but when we combined the two pieces of hardware, we realized we would be able to create a cheap, easy to setup virtual interactive environment.
What it does
KinectVR allows for full room positional tracking of a person, and combined with an oculus, allows for a virtual viewpoint of the room around you. Your entire body becomes part of whatever you create.
How we built it
Using unity integration for the kinect and the oculus rift DK2, we are able to get some basic features out of the box. We ended up having to write a full skeletal tracking system for the player model.
Challenges we ran into
After bricking two computers, we learned that the Oculus drivers do not play nicely on windows 10. A couple hours later (and a couple of system restores) and we were still frustrated with the lack of support on windows 10 for the Oculus rift. With support channels claiming the issue to be hardware related on the laptop side, whereas an older download of the driver would allow us to run demo programs perfectly, but would crash on our code. After about 12 hours of debugging, and testing what we could, we were fortunate enough to find a friend that had the same laptop model as us, but with windows 8 installed. Windows 8 worked a ton better, but we still had an issue of the Oculus blanking it's screen after a few rapid head turns. Surprisingly, a different HDMI cord fixed this issue, even when we knew the Oculus's HDMI cord was fully functional. Along the way, we also faced issues with inconsistent kinect sdk documentation
Accomplishments that we're proud of
We're really proud of how smooth the overall project is. Delving into this idea, we were scared of latency between the kinect and Oculus, creating motionsickness and disorientness that we saw with many Oculus demos. Overall, the program has no noticeable latency between the kinect and the Oculus, and the desktop program is fully usable on an above average laptop.
What we learned
Many of us did not know any Unity going into this project. Each of us can now comfortably say that we would be able to make a simple, if not moderately complex unity program. We also got to use some hardware not typically available to us, and learn the features and advantages of each hardware system.
What's next for KinectVR
Included in KinectVR is some mock ups for an interactive game, as well as a virtual desktop for a more work centered environment. We hope to have the time to expand upon these concepts, and get a fully functional virtual desktop out of what we currently have. We would also, with more processing power and possibly more kinects, would like to get augmented reality integrated into the project. We had some moderate success with this to begin with, but unfortunately lacked the power to do much with it.