Music Genre Classification DL Final Project

• 

  Figure 2. Confusion matrix for one test batch

• 

  Figure 1. Percent correct per genre for one test batch

Classifying 10 Common Music Genres with Deep Neural Networks

Who: Sasha Liu sliu116, Julie Karam jkaram, Nathan DePiero ndepiero, Gavin Sabalewski gsabalew

Introduction: We will be implementing an existing paper titled, “Music Genre Classification using Machine Learning Techniques” (Bahuleyan 2018). The paper’s objective was to create a music classification model based on the genre for the purpose of automatic organization of music libraries - “Being able to automatically classify and provide tags to the music present in a user’s library, based on genre, would be beneficial for audio streaming services such as Spotify and iTunes” (Bahuleyan 2018). The study utilized the Audio set data set, an expanding ontology of 632 audio event classes, and a collection of 2,084,320 human-labeled 10-second sound clips drawn from YouTube videos (Gemmeke et al., 2017). They report an AUC value of 0.894 and a testing accuracy of 0.65 for an ensemble classifier, which combines the two proposed approaches. We aim to implement this model on another audio dataset and achieve similar performance. Music Genre Classification has been done many times, but each implementation uses a unique model. We want to see how this particular model will perform on another dataset. Therefore, we will implement the model from the paper on the GTZAN dataset, which is considered the MNIST of sounds. The dataset contains a collection of 10 genres with 100 audio files each. The dataset contains a CSV file, where for each song (30 seconds long) a mean and variance computed over multiple features.

Methodology: We used mel spectrograms as inputs to the model. Mel spectrograms depict audio files in a 2-dimensional format that replicates how humans hear differences in frequencies, unlike a regular spectrogram. In order to process the 1000 30-second .wav files into mel spectrograms of 3 second snippets we created a script that used the librosa library. This resulted in a 2-d array which we repeated into 3 dimensions to input as a "color" image into the VGG-16 model. We chose to do this instead of converting the mel spectrogram to a .png image with a color to represent the different decibels values across the spectrogram because converting to an image would distort the data being represented by converting decibel strengths to colors. The 3D versions of the 2D Mel Spectrograms are fed into a VGG-16 transfer learning model with pre-trained weights. This is a model (depicted above) that has several groups of convolution layers separated by max-pooling layers and the output of these is run through dense layers for classification. It is designed for image classification where pre-trained weights for the convolution layers that are trained on generic image data can be imported, and using these allows for faster training for different image datasets because the model starts with some feature recognition already learned. By converting the Mel spectrogram into a 3D tensor, it was able to be inputted into the VGG-16 model with default pre-trained weights. We modified the output of the VGG-16 from 1000 classes to 10 classes. This output uses a softmax activation. Thus, the index of the largest probability in this array is the classification of the song. We used Cross-Entropy for our loss function and Adam with a learning rate of 0.001 and a weight decay of 0.0001 for our optimizer. The model was built using PyTorch and trained using Google Colab GPUs.

Results: We were able to achieve an average testing accuracy of 75% and loss of 0.70. The genre that the model was able to identify the best was hip-hop, followed by classical and metal (Figures 1 and 2). Images also shown bigger in the gallery above.

Figure 1. Percent correct per genre for one test batch

Figure 2. Confusion matrix for one test batch

Challenges: We encountered several challenges while working on our project. Our first challenge was becoming familiar with this new dataset. Audio files are something that we have not worked with before, so we spent a lot of time becoming familiar with this new data type. We also had to become familiar with a new analysis package, Librosa, which is a python package for music and audio analysis. It provides the building blocks necessary to create music information retrieval systems. Converting our audio files into spectrogram images which could then be turned into Tensors was a challenging task that we spent a lot of time on. Additionally, we are implementing our project in Pytorch, which none of us had prior experience with. We are familiarizing ourselves with this package, and applying our knowledge and skills from class to successfully implement DL models in Pytorch. Lastly, figuring out the best way to code collaboratively was also a challenge that we encountered.

Reflection: We went into this project with a goal of achieving an accuracy similar to what the research paper obtained. Our specific quantitative goals were to first achieve 50% accuracy, and then target 60%. Our stretch goal was to achieve 70% accuracy. We were able to achieve our stretch goal, as we obtained an average accuracy of 75%. Our model did work out the way we expected it to, but it was interesting how accuracy varied between genres. Hip hop obtained high accuracy, while jazz was lower. This could be attributed to the different structures of these two genres, with jazz often having improvisation sections leading to less similarity within the grouping. It was fascinating to see how different models can perform differently based on the dataset that it is trained on. While there are many factors that could contribute to our higher accuracy than the paper, we believe that the dataset we trained our model on contributed to our success. Overall, from this project we were able to learn more about the VGG16 model, and were able to recognize the utility of VGG16 to tasks other than image classification. Time-series data, such as spectrograms, can also be used for this model architecture. In our first approach when building the model, we tried converting the audio files into images, but then pivoted to creating spectrograms to represent the audio files, as an image distorted the decibel representation.. A limitation of this model is that the dataset we used does not include all existing genres. More robust genre classification may be necessary as music genres become more niche and music streaming platforms compete to give the best recommendations to their users. Future work may look to classify songs into more specific sub genres and may even use lyrics in conjunction with spectrograms for this purpose.

Github link: https://github.com/natcasd/DL-Final-Project.git

Poster link: https://www.figma.com/file/HAPI1rZj5cYVl7tUJw30ZP/DL-Final-Project-Poster?t=f0LdIhPwuskxsFji-1

Built With

• python
• pytorch