PyVillage YMCA Detector

Inspiration

In these unprecedented times, some lighthearted fun is required to maintain sanity. We discussed as a group, that there are very few people who walk the face of the earth who dislike the YMCA, the dance, and the song at least. With this in mind, we realized that some people may not know whether or not they're doing the YMCA. We wanted to come up with a solution to this devastating problem that is tarnishing our society.

What it does

The PyVillage YMCA Detector will detect which letter of the YMCA you are making with your body, and if you do all the letters in the correct sequence, it will actually play the YMCA for you to dance along with!

How we built it

Using Google's MediaPipe and OpenCV, we were able to receive several points from a person's body. With those points, we were able to apply some trigonometry in order to calculate the angles that the person is making with their arms, and we used that math to calculate which letter they are making. We then used OpenCV to build out the rest of our UI and used a small library called "PlaySound" to play the YMCA if the computer detects that you've successfully completed the YMCA.

Challenges we ran into

The first challenge we encountered, was gathering training data. There was not a ton of data on the YMCA out there, so we reached out to everybody we know on social media and asked them to send us pictures of them doing the YMCA. We sadly only got about 30 responses, which is simply not enough data to train a neural network on. We solved this, by instead using Google's MediaPipe, which allowed us to get points on a person's body, which we could then apply trigonometry to in order to mathematically determine which letter the person is doing if any.

The next problem we faced, was that originally, we wanted to have our program connect to a Google Home device if it detected one was on the network, and play the YMCA from the Google Home device. We were able to successfully connect to it, however, we were not able to properly send the file across. We stopped trying that due to time constraints and simply had it play from the computer's selected speaker device, however, we learned by doing that that our mp3 file was corrupted, which would have explained why that was not working. At that point, however, we didn't have enough time to re-implement the Google Home functionality, and as such, it is still playing through computer audio.

Another challenge we ran into, was properly installing our libraries. For some reason, MediaPipe does not work on Python versions >= 3.10.0, and as such a lot of us had to switch our Python versions. This was quite the process, especially for the Windows users on our team.

The last challenge was figuring out threading in Python. We started the project by building the letter classifier and then added music functionality later. The problem we faced with this, was that the music would take up our resources, and not allow the classifier to run while the music was playing. We fixed this by implementing threading, and having the music play on a separate thread, however when we tried to exit the program, we would have to run the command twice to stop the whole program. We fixed this by setting the music thread as a daemon so that when we exit the main program, the thread also aborts.

Accomplishments that we're proud of

We are very proud of our ability to adapt and overcome in very short amounts of time. Completely restructuring our process was not an easy one, but a required step due to our lack of training data. Google provided excellent documentation on how to use their MediaPipe model, but we still had to spend some time getting familiar with both MediaPipe and OpenCV, something that none of our team has ever used before.

What we learned

The biggest thing we learned, was that even after listening to the YMCA over 100 times, it still doesn't get old.

We also learned about the process involved in using a premade model for our own purposes. This is something none of us have done before, as in the past we have all created our own models for whatever purpose we need. This will prove incredibly useful in the future, as there are quite a few models out there that do phenomenal things that we couldn't dream of building ourselves.

What's next for PyVillage YMCA Detector

Our next step is to try and implement this into an app. We briefly looked into Apple's CoreML kit, which provides quite a few very useful resources that would help us a lot with this, however, we will need to learn Swift in order to properly do this.