We share a love for music and a love for cool gadgets. We saw the leap motion device, and we thought we could create an innovative program to play instruments in virtual reality. Along the way we also thought it would nice if the program could transcribe the music that the program output into sheet music.

What it does

Detects hand motion to simulate playing piano and drums. It can also convert the music output into sheet music for drums.

How we built it

We used a leap motion device along with python.

Challenges we ran into

We ran into many challenges in learning how to use Leap Motion's SDK. We also faced several issues in fine tuning the recognition of certain motion. Lastly, we had trouble developing the transcription part of the program.

Accomplishments that we're proud of

We are all First-Years and this is our first ever Hackathon. The first day we were extremely confused on what to do, but now we are proud to have developed a functional product. We never thought we would be able to transcribe music played, but we exceeded our expectations in this way.

What we learned

We learned a lot about using the Leap Motion device, and we learned a lot about sound in python.

What's next for VirtuBand/Transcription Tool

We want to expand this program to allow for multiple instruments including guitar, violin, and trumpet. We also want users to be able to play with their friends online or in person. Lastly, we want to improve the transcription tool so that it can produce full scores from user interaction with the program.

Built With

Share this project: