Towards the beginning of the hackathon festivities, we spent a considerable amount of time playing around with Spotify's infinite jukebox. After uploading more songs than we care to admit, we noticed that some of the songs that worked well with the jukebox were composed of similar (nearly identical, even) sections. Inspired by the jukebox's ability to seamlessly seek within songs and still maintain musical believability, we were inspired to try our own hand at finding similarity between song sections. However, instead of using the similarity to create infinite loops, we set out to reuse repeated sections as a form of audio file compression.
How it works
First, we upload an mp3 file to echo-nest. After waiting a few moments for the server-side to perform audio track analysis, we then fetch the results and calculate similarity between all song "segments". Then we choose the most similar segments and group "runs" (sequential occurrences) of the segments into song sections. Afterwards, we compare the sections to determine which sections are "equivalent" to one another. Lastly we strip out unnecessary sections from the input audio file and output a series a necessary song sections, along with playback info, that tells a Python script how to interleave the sections to recreate the original track.
Challenges I ran into
It was really difficult to figure out exactly how to break apart a song and still maintain musical integrity. Similarity values between song segments aren't entirely reliable, so we had to be smart (and conservative) about how we tried to group things together. Also, we were only allowed so many requests per hour, so it was difficult to test our system to make sure it was working properly.
Accomplishments that I'm proud of
We did it. We implemented a novel compression technique using cutting-edge song analysis information. It's really cool to hear the song interleaved together!
What I learned
How to perform HTTP requests. How to deal with JSON data. How to deal with "data in the wild". Have to use echo-nest for track analysis (we'll be doing a lot of this in the near future - it's so cool!)
What's next for close-enough
NOTE: The close-enough project image is not ours - we got it from paul@spotify (http://musicmachinery.com/2012/11/19/visualizing-the-structure-of-pop-music/)