OcuLeap: A new way to explore

Inspiration

Our main inspiration in join the hack-a-thon was to try something new and work with cool technology that we don't get the opportunity to interact with. Our team comes from a primarily scientific computing background so we were interested in branching out into more application based projects. One of the most exciting areas of technology today are the new and incredible developments in how we interact with computers; the Leap Motion and Oculus immediately stuck out to us as the best way to explore new technologies and create a really cool product. One can imagine the incredible ways education and research can be improved with such immersive digital experiences.

What it does

Generally, our project is about exploring how to combine virtual reality gesture controls with common computer activities. We did this in the form of implementing the ability to control virtual objects with our gestures. We specifically did this in two ways; the first is a port of the famous angry birds game. The angry bird setting is moved to 3D virtual reality and the birds are controlled with a series of hand gestures. The second project is an effort to explore how scientific computing, specifically graphing and visualization, can be enhanced with new immersive technologies. The idea was ti construct a series of common scientific visualizations: scatter, time series, histogram, multi-variable functions and then implement the ability to manipulate and explore them. Practically this resulted in some awesome looking animated graphs and the functionality of being able to rotate them one hand and scale it them with the other.

How we built it

The core of development centered around the unity game engine which provides the interface and IDE, Monodevelop, for developing new digital worlds. Both Oculus and Leap Motion provide resources for integrating their respective functionalities into unity.

Challenges we ran into

First and foremost we had performance issues when Oculus software determined an Alienware laptop didn't have sufficient resources to support it. Luckily we plugged it into a Macbook and it immediately worked. Unluckily, development on OSX doesn't seem to much of a priority and we had to spend a significant amount of time finding the right combination of the latest version of unity and depreciated Leap motion SDK in order to get the full setup running. After that it was a matter of overcoming the learning curve for unity and the sdks for the equipment.

Accomplishments that we're proud of

First off, we're just happy we were able to successfully integrate the Leap Motion and Oculus environments with unity, we are especially happy that we were able to accomplish an interactive, gesture-controlled 3D work and fun project. Mapping the location and velocity vectors of the hand from the Leap Motion onto objects in the virtual world was a difficult but extremely satisfying task.

What we learned

Almost every aspect of this project (outside of programming in general) was new to all of us; we gained a great amount of knowledge about how game engines work and how to implement new functionality and behavior into them. This was our first hack-a-thon and so the experience of choosing a project and working together to accomplish (a most probably over ambitious plan) in a short amount of time was instructive in itself.

What's next for OcuLeap: A new way to explore

We'd love to see teachers can use virtual-reality world to teach; it really will affect almost all aspects of society. Also, we now want our own Oculus and Leap Motion so we can continue to make fun projects!