We wanted to integrate hardware into our project and after spending some time looking through the options at the hardware lab we discovered the Leap Motion hand tracking sensor. We thought it would be really cool to use hand gestures to control some sort of device with Arduino and ended up using it to control sound to make an "Air Piano."

What it does

When you place your hands over the Leap Motion sensor, you can play different tones by moving one of your fingers down. Each finger is associated with a certain tone that gets sent from one Arduino to another Arduino that has a Piezoelectric buzzer that plays the according tone.

How I built it

Using the Leap Motion SDK and Python API we extracted useful information to track the location of each finger tip. From this information we calculated the max downward change in distance out of all the fingers that was above a certain threshold to find the finger that is "pressing down a key". Then, using the pyserial library we send a char to one Arduino which sends the according frequency to another Arduino with an RF transceiver. This second Arduino then plays the note using a piezoelectric buzzer.

Challenges I ran into

The main challenges we ran into were configuration and setup of different libraries. For example, the Leap Motion SDK had a bug with Windows 10 that needed a manual fix that was documented in the Leap Motion forums. Also, it was a little tricky to figure out the pyserial library and how to use the RF transceivers, but after configuring everything we were able to start programming the logic.

Accomplishments that I'm proud of

We are proud of being able to integrate multiple different things into our project from the Leap Motion sensor to Arduino.

What I learned

We learned how to use Leap Motion sensor and I personally learned more about music and how you can create different tones using the buzzer.

What's next for Air Piano

Our main goal is to increase the accuracy and precision so that we can get cleaner results. Also we want to add a feature that would allow you to put multiple fingers down at the same time. Once we perfect the Air Piano, we could move on to other instruments.

Share this project: