Some group members were highly involved with piano in the past. After talking about experiences with the instrument, three main problems with it come up. First, it is a very hard instrument to move around with you (to university etc.) making it hard to remain practiced. Second, it is impossible to play long with people who are not sitting beside you on the same piano. Lastly, the piano is an extremely expensive instrument which prohibits some people from ever having a chance at playing one. Creating a cheaper alternative would allow more people to become familiar with the piano. With ε-Harmony we solve all these problems.
How it works
The leap motion is used to detect the position and orientations of the hands used to play the instrument. This allows a virtual set of hands to be created. A custom environment (designed in Solidworks) is setup in a virtual reality with a piano in front of the user and they simply play as they would in the real world. Another user who is paired on the same server can access the same piano and they can create music in the virtual world together.
Challenges we ran into
Throughout this project, many system integration problems came up. It is worth noting that nobody in our group had ever used Leap or the Oculus rift. So compatibility was determined more or less through trial and error. More specifically, we encountered issues using the leap with different versions of our development environment (Unity). This made us question the reliability of our hardware. We found the best way to handle this situation was to split the group and allocate one group member to become an expert with Leap, and one with Oculus rift. After this decision was made, systems were integrated fairly easily.
Accomplishments that we're proud of
- Built the app from the ground, acquiring new coding skills along the way
- Learned how to interact with two completely new pieces of hardware (Oculus and Leap)
- Created an app that seems like something people would truly use
- Solved the main issues with piano playing in a single application
What we learned
Leap motion is extremely reliable, and the environments that can be created in Oculus are breathtaking. However, if there is any one thing we've learned it's that piano is not our forte.
What's next for ε-Harmony
Possibility for more instruments, touch ups on the quality of play, touch ups on multiplayer.