Lots of people are creative enough to write awesome lyrics, but they lack the musical knowledge to create songs with them. What if we had a program to generate a good song depending on the text input? That's exactly what AutoDJ does.

What it does

Using Microsoft's Text Analysis API from Cognitive Services, we analyse the key-phrases of the input text given by the user. We also look if it is happy / sad, using the sentiment analysis feature of the API. We compare the resulted features against a large collection of song lyrics, using the cosine similarity measure. The first few best matches are then fed into a Restricted Boltzmann Machine algorithm that generates a new song from them.

How we built it

First we downloaded a large number of popular songs from YouTube (the instrumentals only) and their lyrics separately from different websites. Then we have a script that calls Microsoft's Text Analysis API and gets the key phrases for the user input, and compares it using the cosine similarity measure with the key phrases from all the lyrics downloaded. We take the instrumental songs of the best matches (encoded as .mid) and feed them into the neural network that generates a new .mid with similar sounding.

Challenges we ran into

It's very hard to generate new music, as this is a cutting-edge current research topic. It is also very hard (and thus unreliable) to convert mp3s into midis. We had some troubles doing that, and even now it's not too accurate. Of course we also had the usual issues with permissions, dependencies, etc., but who doesn't have those ?

Accomplishments that we're proud of

Actually managing to complete a pipeline user -> algorithm -> result -> user.

What we learned

Quite a lot. Mostly about neural networks and language processing.

What's next for AutoDJ

Some extra features could be implemented such as improving the accuracy of similarity, by taking more details into account. Also we could have more than just happy/sad sentiments, and maybe even analyse the main topic of the text.

Built With

Share this project: