We wanted to generate jazzy progressions, made interactive by a person's movements.
Our idea was to train a sequential generative model for music. Instead of waveforms, music is encoded as text (e.g. Cmaj7). This is a smaller search space (roughly 1000 “words”) instead of 44100 Hz samples: one beat per word with any given time signature. Text can be reliably converted to MIDI for playback depending on its format (ABC is the standard).
How it was built
Pytorch, lots of Bash, Python scripts.
Music data is a mess. Formats are not very well documented, and the repositories that exist for jazz music are in many different formats, few of which are ABC. Most links are dead!
In the end
We learned a lot about music notation, software surrounding it, and it sort of works! It's necessary to have a standard, clean data set. That should be worked on. Music playback should be live. Code should be refactored. When all of that is done, Jazz Hands will be complete!