The provided hardware and our shared interest in the project's implemented fields led us to create our piano simulation/game. The ability to actively interact within a digital space in the same manner we would in our everyday lives inspired us to create our piano simulation. Playing a piano is prominently known for its necessity of long individual fingers and high levels of hand eye coordination both of which lack in most main stream VR experiences. By implementing the LEAP Motion sensor our team was able to chip away at that barrier between the virtual and actual reality, driving us to to continuously work towards creating this demo despite our delusions at 4 am from lack of sleep (which we still never gained due to our sheer passion to polish this project to the best of our abilities).
What it does
Press a button to activate the countdown that starts the game. Every time a note on the keyboard flashes red, the player must press down on it to play the note. This carries on until the song is finished. Besides the educational game that shows users how to play "Twinkle, Twinkle, Little Star," in the virtual space the piano is fully functional with moving keys and as accurate sounding as we could get in 24 hours.
How we built it
We had to connect the Leap Motion Controller and the HTC Vive to Unity. We then had to set up the models for each key, connecting a piano note to each one. Eventually, it was time to program the keys to flash red, signaling that the user should play that note, and to write the included song. Every motion, sound, and event was accomplished using the script system within Unity, which is comprised of C# files. Numerous tests of key and button interactions were performed in order to provide the best piano playing experience.
Challenges we ran into
Most members had not used Unity, C#, much less combining them with VR. The one member, who had prior experience, found difficulty in combining the three. Learning and implementing what we learned simultaneously to create our VR experience proved quite difficult and led to numerous errors from pointer errors to plain old compiler errors.
Accomplishments that we're proud of
We are most proud of being able to create a fully functioning piano and using the Leap Motion sensor to model individual fingers instead of preset controllers to maximize the user's ability to create music.
What we learned
We learned how to use the HTC Vive and the Leap Motion controller in conjunction with Unity. In addition, most members had their first experiences working with Unity and the C# language.
What's next for VRpL
Various features were proposed for future addition, such as a system of lives, other songs or instruments, multiplayer, and tracking improvements.