Air Piano

Have you ever wanted to play the piano but didn't have one? Or maybe you can't lug one around? With our project, you'll be able to play piano wherever you want! Just set up the base station and play.

Comment

Inspiration

One of our team members is a big piano enthusiast, and was wondering whether we could quickly create a personal piano that can be played anywhere.

What Air Piano does.

Air Piano uses accelerometers and computer vision running on the edge with an Arduino Uno Q to detect which finger is playing which key with how much force. Then, a piano synthesizer made from scratch generates audio played over Bluetooth into headphones that the listener is wearing.

How we built it

Jash and Abhijay dealt with all the hardware (wiring up sensors and creating CAD), Ishan worked on audio (synthesizer and Bluetooth), and Rushil worked on computer vision and integrating all the elements into a finished project.

Challenges we ran into

We initially meant to attach IMUs to every finger. Unfortunately due to manufacturing issues and low quality hardware we had to use the one IMU that actually worked. Additionally, since the Arduino App Lab runs in a containerized environment, we had to craft a way to communicate with the host operating system so we could run audio over Bluetooth.

Accomplishments that we're proud of

Our computer vision model running directly on the Arduino is very accurate with respect to positions.

Our piano sounds quite real! We used the Fourier transform to interpolate between and extrapolate beyond three samples of a real piano from Columbia University: https://www.ee.columbia.edu/~dpwe/sounds/instruments/.

What we learned

We learned how to use computer vision on the edge, the mathematics of music, and the way to use IMUs to detect taps. We also had to do a lot of plumbing work to get this working on the Arduino Uno Q platform.

What's next for Air Piano

We want to extend to all fingers, and we would like to add in more instruments, which can be navigated between by moving your hands forward and backward.

Built With

Share this project:

Updates