I've always been interested in Generative Art and the possibilities of creating art using machine learning. With music generation models getting better and better I wanted to try to build something with them, and I wanted to combine a visual component as well. And so the idea for Electric Sheep was born, what would the daydreams of a bored AI look and sound like?

What it does

Electric Sheep is a web application that plays AI-generated music, accompanied with a unique visualization for every song. New music is continually generated so you'll never run out of music to listen to!

How we built it

The backend of the app consists of 3 Docker containers all using Python: one which generates the music, another which processes it, and a Flask server container to serve the music (and provide a simple API). It uses Google Magenta's Music Transformer model to generate the music (using the pre-trained weights). The frontend is plain-old Javascript, HTML, CSS. I used p5.js for the visualization.

Challenges we ran into

  • There was some difficulty getting the Music Transformer Tensorflow model running in a Docker container
  • I struggled getting HTTPS working on the Flask server and didn't manage to get it working in time (so I couldn't host it on my own domain which uses HTTPS)
  • It was tricky getting the UI looking good on different displays/aspect ratios

Accomplishments that we're proud of

  • Endlessly generating music is awesome!
  • Good looking website!
  • AI pun song titles

What we learned

  • I learned a lot about using Docker, especially with more than one container running at a time and interacting
  • I learned more about transformer models
  • Learned more about p5.js

What's next for Electric Sheep

  • I'd like to train a custom model for the Music Transformer
  • Add more to the visualization (something in the background maybe), make it more distinct for different songs
  • Song rating system, option to play the highest rated
  • Get it running on my own website instead of a random VM

Built With

Share this project: