The Infinite Melody

A website using an RNN AI to play a constantly updating piece of music, trained on a database of chords extracted from scraped midi files.

Comment

Inspiration

We wanted to see what we could produce with AI infitnite generative media, and seized upon music as a great way this can be demonstrated and experienced first hand.

What it does

We have created a website which, when loaded, starts playing music generated by our AI, which will continue to generate and play original music (hopefully) indefinitely... Along with this we have a satisfying sheet music animation showing which chord is being played at any time.

How we built it

  • We started by scraping several public websites for thousands of midi files and downloading them.
  • These files were then converted into python objects using the Music21 library and an algorithm was written to extract the key information from these objects, this being: isolating the piano part of the song, finding the tempo at each point in the run, collecting all the notes and chords and their duration and offset within a measure.
  • The output was a 2D array with 2 columns for each song, the first being chord (including rests) and the second being duration.
  • This was used to train an RNN AI which could produce it's own list of chords and durations.
  • We tested the AI output using a function which could convert these 2 columned arrays back into midi files and used this to refine our RNN instructions.
  • A website was developed using javascript and HTML which could could play the live updating melody and produce a paired scrolling sheet music animation.
  • A flask backend is used to host the server and send the updating music array to the javascript front end.

Challenges we ran into

  • Not enough music theory to properly understand the Music21 library.
  • Complexity of midi files resulting in many oddities in the database which had to be mitigated.
  • Lack of experience of Javascript in the team so lots of learning on the job needed for the front end.
  • Over 27000 unique chords resulted in a lot of different weightings and values necessary for training the AI.

Accomplishments that we're proud of

  • The AI can actually generate music which doesn't sound terrible!
  • Successfully extracted over 500,000 chords from over 600 scraped pieces of music for the use of training the AI.
  • Playing live sound and animations in javascript (none of us who worked on this had ever attempted javascript before).
  • The sheer number of chords we have in our database allows the ai to generate for an uncountably long time before it risks repeating itself.

What we learned

  • A basic knowledge of music theory - I've learnt more about music at DurHack than in the rest of my life.
  • Javascript for the 3 members of our team who have never touched it before.
  • First time using RNN AI.

What's next for The Infinite Melody

  • Increase the size of the database.
  • Increase the time we let our AI learn from it.
  • Try and extract more useable data from the midi files (look at other instruments, time signatures, etc...)
  • Make the website more aesthetic.
Share this project:

Updates