MelodifAI

• 

  Flask webApp

The Problem

The great classical composers of the past are no longer around to create new music, but what if we could mimic their genius using technology? The MelodifAI project uses deep learning techniques to bring a new twist to the old classics by generating new, original piano music based on existing works of classical composers.

The motivation for this project comes from wanting to try mimicking human creativity using artificial intelligence. This is a beginner's project, so the goal is not to create a perfect result, but rather to explore to what extent AI tools can be used to create something that many would regard as only possible by humans.

How we built it

At the core of MelodifAI is an LSTM network model. It uses as input a sequence of notes and attempts to predict what note is most likely to follow. An Long Short Term Memory network is used because the data is time-dependent (ie. playing the same notes in a different order changes the music completely). Each predicted note is used as part of the input of the next iteration, so that the music can build on itself. This process of generating notes one at a time based on the preivous ones can be continued indefinitely, meaning that there is no limit to how much music the model can be used to generate.

Challenges we ran into

The main challenge of this project is pre-processing and post-processing the data. The information of each note from the dataset needs to be extracted from a .mid file and encoded into strings representing each note. The model takes into consideration the pitch and duration of each note, as well as chords made of multiple notes. After the predictions are made, the process needs to be reversed to convert the output sequence back into a playable audio file. This encoding and decoding process was the most difficult to implement. Our method is based on a notebook by Karnika Kapoor (link), which we built upon to include duration encoding and methods to simplify the dataset while preserving as much as possible the artistic identify of the music.

Accomplishments that we're proud of

The generated music from the algorithm is by no means a masterpiece, but we are happy to see that the model did learn some aspects of music composition. It is very interesting to see what type of knowledge can a computer learn about music simply by looking at lots of data. Without any prescribed knowledge about rhythm, notes and chord progression, the algorithm found its own unique approach which is often not very musical but can shine in some moments.

What we learned

The most valuable lesson learned from this project was that AI is not a perfect tool that can solve any problem. Although deep learning can be extremely effective in some scenarios, a lot of though and care must be taken to use it effectively. In MelodifAI's case, our approach was a little too primitive to create high quality music, but that does not mean that this problem cannot be solved using deep learning.

What's next for MelodifAI

To further improve on this project, we could rethink our encoding algorithm to take advantage of similarities between notes. Just like how word embeddings are used to encode closely related words similarly, a similar approach could be used to encode related notes similarly, giving the model a better notion of how its search space is structured.

Built With

• flask
• keras
• music21
• python
• scikit-learn
• tensorflow