Ever wish you could lug your grand piano to university and impress everyone with your take on Für Elise? Well, we have. While the piano is a beautiful instrument, it isn't always the most portable. With this technology, anyone can play piano anywhere. All they need is their phone and our luxurious gloves. It's a cheap, space-efficient solution to play music on the go.
What it does
VRtuoso creates an experience where a user can play on a piano in mid air and on the go. The user wears the Google Cardboard to view the 3D piano from our Android app, as well as our VRtuoso gloves™ built with accelerometers on the fingertips. The user presses the virtual piano keys, and the corresponding sound is produced from the Android app. In this way, users can practice anywhere they are, without needing the physical space for a real piano, and without needing to be able to afford a real piano.
How we built it
The project was built using Google Cardboard, Unity, Arduino, as well as a variety of hardware components including Grove - 3 axis digital accelerometers. In order to create a virtual piano, accelerometers were sewn onto the fingertips of a pair of gloves and data was exchanged to the Google Cardboard using the Arduino. Unity was used to design the VR piano mechanism.
Challenges we ran into
When building the piano with movable hand in Unity, we first decided to make sound by collision of the fingers and the piano keys. However, it was hard to simulate the hard pressing down gesture in Unity. Therefore, we came up with an alternative solution by rotating fingers and making sound by comparing the positions of fingers with positions of the keys. Each accelerometer must be connected to one Arduino. We don't have enough Arduino and it will make the project too heavy,
What we learned
It is the first time for all of us to use Google cardboard. We learned the benefits of dividing work based on each team member's strengths. Part of our success was because we were able to help each other as a team and work independently to create solutions to our problems.
Accomplishments that we're proud of
We're proud of how many new things we tackled that we've never tried before. For example, nobody on the team had ever worked with Google Cardboard before. Some of us had also never developed Android apps before. We challenged ourselves to take risks and reach new extremes.
What's next for VRtuoso
VRtuoso can be extended to work for other large instruments like bass, harp, drum kit, and even a tuba if we implement a wind sensor.