Dynamic Vector Based Spotify Recommendation Engine

Inspiration

Last year with the emergence of LLM we got interested in Vector Databases and since then we have wanted to build one but it seemed to be a daunting task. One of our other members was working on another project concerning wave processing and we had the idea to create a music recommendation app. Since vector databases can theoretically be used for recommendation we decided to dive in and build this project.

What it does

It looks at the users recently played songs and then classifies them into a mood. It then gives the user recommendations on songs that are similar to that mood

How we built it

We used the Spotify API to get information about a song such as the isrc. Then we used the src with the MusixMatch API to get the lyrics. We ran sentiment analysis using a HuggingFace model on the lyrics of songs to get 6 basic emotions. From there we created 21 new emotions using linear combinations of our basic 6 and classified each song into these emotions. We represented these emotions as a vector in a vector database. Using an algorithm called KNN based on pair wise cosign similarity calculations we were able to find vectors with similar sentiments.

Challenges we ran into

Most of our challenges came from conceptualizing and understanding the theory behind vector databases. However by far the most frustrating part was dealing with the APIs. Their documentation was completely lacking and their errors were nondescript. Finding workarounds for these issues was where a lot of difficulty arose.

Accomplishments that we're proud of

We are proud of the fact that starting off we had no idea how vector databases worked in practice and when we finished we all have confidence that given the circumstance we have a similar degree of confidence to use it as more established databases. We were also proud of the fact that we created a full fledged product. There are definitely things we can add and improve given a longer timeframe but the framework for that has been set up.

What we learned

We learned how vector databases work and how to implement them. We also learned how to use multiple APIs and how to coordinate all the data to create a good user experience.

What's next for Dynamic Vector Based Spotify Recommendation Engine

There were a few issues with the Spotify API that the developer team has said they will address. After they do we can also recommend songs that are similar sounding using the same database. In the future we can also combine these two databases and implement a gradient descent model to more accurately optimize the weights for each parameter.