We came up with this idea in October 2016, when I saw a team made an Artificial Intelligence that composes poems.
What it does
Our first AI: Machine Learning Based AI: It takes in a short sequence of notes, called "primer sequence", and spits out a much longer melody. The primer sequence can be as short as one single note. If you don't want to input a primer sequence, a sequence will be picked by AI.
Our Second AI: Music Theory Based AI: We also have another music composing AI, which takes no input at all and generates music base on modern music theory along with advanced chord progression with a set of instruments. Also if we provide the root notes along with the chords, the A.I. can also produce music base on user input.
EX: We also use leap motion for awesome user interaction. We also used Amazon Echo to maximize user experience.
How we built it
Our first AI, the one that takes input and generates a sequence of melody, is built in Recurrent Neural Network with python. Unlike traditional deep neural networks, which takes the input all at once, an RNN takes input chunk by chunk. During each iteration, the RNN will make a prediction about what and when will the next note be. That prediction along with next chunk of input �will be fed back into the RNN for computation of next round. After all, inputs are processed, the sequence of all outputs is the generated melody. Now the modeling is done. We then used hundreds of classical piece to train the AI. This part is done with genetic programming with various algorithms for optimization.
For the second A.I. we "taught" the A.I., which involves a lot of research, some of the modern music theory to give it a headstart. Then we attempted to refine the algorithm to change the way it generates music to make it sound more harmonious and better. We also built a module that reads other music files and analyzes their chord progression and melodies. Also, we tried to combine visual and audio to generate music from given photos.
Challenges we ran into
The knowledge about music. How to translate music theory into binary. How to parse midi files. How to model an AI and train it. How to work with VR devices. How to communicate between two programming languages.
Accomplishments that we're proud of
We managed to build a music AI that produces decent music in Jazz genre.
What we learned
Not sleeping for two days is bad for health.
What's next for Music ex Machina
We will keep improving the AIs and they will be very useful for DJs, music producers and singers for music composing with the knowledge they know from learning.