Inspiration
It all started when I was 5 years old I was always fascinated by instruments specially drum and the musicians who play it. When I got to know about virtual instruments which are available on websites when I was 12. But I want to physically move my hand and not click or touch on keyboard and mouse so I used opencv to make my finger drum sticks.
What it does
When user will run the program webcam will open and on the basis of number of fingers which are open it will play the sound, but there is one condition, to play the sound produced on 2 fingers after opening 2 fingers you have to open 0 fingers or fingers other than 2 ,and you can use only one hand.
How I built it
I built it using PYGAME, OPENCV, 𝐂𝐕𝐙𝐎𝐍𝐄, 𝐌𝐄𝐃𝐈𝐀𝐏𝐈𝐏𝐄 and NUMPY. Pygame is to make the screen. Opencv to get video through webcam. Numpy is for flipping video from webcam. CVZone and Mediapipe for detecting hands and number of fingers.
Challenges we ran into
I was not able to connect pygame with opencv , and there was a lot of issue in playing music in background.Detecting number fingers open was hard too.
Accomplishments that we're proud of
I am proud of that I am able to sumbit project at time because, First I created whole project using opencv but at the time of playing audio in background the video from webcam lagged alot making my whole idea worst. Then I created project with pygame and opencv. I am also proud of that I am able to write all this content on devpost successfully cause It took me 1/2 hour to write this but I did not save it and accidentally I closed the tab.
What we learned
I learnt a lot of new things in opencv and pygame. I also used mediapipe and cvzone ,module created by my opencv teacher for first time.
What's next for My Finger Tips Are Drum Sticks
I would like to make a good ui and also like to add new instruments with new gestures. not only hand gestures but many more. I would also like to make a web version for it.
Log in or sign up for Devpost to join the conversation.