Music therapy involves the use of musical interventions to help form therapeutic relations between a patient and the music. The music is meant to help address physical, emotional, cognitive, and social needs of the individual. However, we realized that music therapy is still a developing field, and patients throughout the world may not have access to such options of therapy. Thus, we created Smartify to showcase some aspects of how we can use existing technologies and apply machine learning to create therapeutic experiences with music.
What it does
Our app Smartify uses machine learning to predict a genre of music. However, with a very extensive network of genres and diversity found between songs of the same genre, it is important to recognize that predicting music relies on much more than finding songs from the same artist or genre. To solve this, we used Spotify's Open API to analyze attributes of each song. These attributes include the tempo, energy, danceability, loudness, liveness, valence, duration, acousticness, and speechiness of each song. We use deep learning to then discover relationships between the genre of these respective songs and their attributes, and predict other songs based on the discovered relationships.
How we built it
We built the app using Tensorflow, Spotipy, Python, and Scikit-Learn.
Challenges we ran into
Being our first deep learning project, we had to learn the fundamentals of machine learning as we went. Learning how to manage the testing and training data such as batching and shuffling was difficult to understand in terms of the open API and scraping for data to directly analyze.
Accomplishments that we're proud of
We are glad we were able to successfully implement an AI to analyze data of such a large mass and help introduce people to the idea of music therapy.
What we learned
We learned the capabilities of Tensorflow and how extensive and intuitive it is to apply AI to a product using the Keras library. Furthermore, we introduced ourselves to the application of cognitive science using computer science.
What's next for Smartify
This application only serves as a pioneering concept of the idea that we can use machine learning to help music therapy. In the future, some effective advancements would be an increase in size of the training data, as well as analyzing other aspects of a track, such as the album cover and artist's background.