We love music and since Arvind can't play any instruments we wanted to let him know what it feels like without actually learning anything.
What it does
It shows a set of instruments and lets the user control their volume with hand movements and sharp glances at the instruments.
How we built it
The virtual reality part is built with Unity engine and ran on an Android phone using Google Cardboard, the music was produced in different tracks by a former team member that we don't speak of anymore. Unity communicates with the leap motion through a web service, by constantly pulling data that is being posted by a JS script listening on the inputs from leap motion. If this seems off, you're right.
Challenges we ran into
The main challenge was getting the unity app communicate with the leap motion. Since' we're all pretty bad at networking and bluetooth is a deep forest we didn't dare to go into, we decided to use an intermediary for the communication, which ended up being a web service.
Accomplishments that we're proud of
We made it work! It's not perfect by any means, but it works and that feels good.
What we learned
There's always a way to make something work, even though it might not be great, but it's a good start and can be improved later.
JSON parsing can be more difficult than it sounds
What working with Unity engine is like
Don't try to combine leap motion with unity on android anymore.
What's next for Visual Orchestra
- More instruments! More tracks! A live show one day.