We are all piano music lovers. And we figured that people should all have access to a Piano given how expensive they can be and the fact that everyone has a smartphone & laptop so they can use their smartphone or webcam as a camera to play the piano.
What it does
You take a camera and a sheet of paper with drawn out keys evenly distributed across an octave and then you can play the piano with your fingers and then in the future you can play songs and have it be compared to songs and give the user feedback and even more later on have more data analytics tracking the number of key strokes the user has, what sort of music they enjoy, their improvement over time, which songs are most difficult along with the category among piano players. There could be like a Guitar Hero like ranking system where you can compare yourself to others.
How I built it
For our primitive protoype, we build a mounted cage with a webcam looking down in the middle, and then we have a board or a sheet of paper with evenly distributed rectangles where notes are mapped to via Python scripts that use the OpenCV computer vision library to detect if there is a finger on the surface and that change in pixel density causes a key to be played via the windows sound package. Since we didn't go into the full depth data analytics, our only test song is Hot Cross buns which is a beginner easy song, While the user plays keys, it records which key is being played and appends it to a python list and then at the end the song and the users played strokes are compared and it tells you what keys were different and a percentage score of how many key strokes out of the expected key strokes were right.
Challenges I ran into
Trying to configure OpenCV and have it detect finger movements and learning Computer Vision
Accomplishments that I'm proud of
What I learned
What's next for CameraPiano
Have it be more scaleable, have it be able to do multi finger playing, advanced data analytics as mentioned above, lower response time, better frame rates