SeeMyFeels

Kanye West - Gold Digger
Megadeth - Holy Wars ... The Punishment Due
Pharrell Williams - Happy
The Beatles - Here Comes the Sun
Earth, Wind & Fire - September
DonMsc - Lovely (LoFi)
Relaxing Beats & Deep Sleep Meditation - Winter In F Minor
Flo Rida - Low
Lucenzo Feat. Don Omar - Danza Kuduro

Inspiration

We really wanted to explore the more creative and artistic side of computing, which is often overlooked in favour of its more prevalent scientific uses. We had an idea that we found really exciting, and we rolled with it. We are very pleased with the outcome!

What it does

SeeMyFeels takes as input an audio file, analyses its characteristics using a pre-trained neural network, and then uses those characteristics to generate images that we feel represent that music.

How we built it

We analysed the given audio file with Essentia, and then a TensorFlow model, giving scores to 5 predefined classifier tags - Happiness, Danceability, Aggressiveness, Chillness, Acoustic. We then used these scores to code for certain features of the generated images. We tried to tweak the effects of each tag in order to make the image "look like the music sounds" as much as possible. The generative art algorithm consists of particles tracing paths as they move through space, being attracted and repelled by a variable number of 'magnets'.

Challenges we ran into

We would have liked to create a web application that would have allowed a user to upload their own mp3 files, and retrieve the produced image. Unfortunately, we ran out of time as we were unfamiliar with a number of the tools/frameworks that we decided to try out, such as React and Flask.

Accomplishments that we're proud of

We are very pleased with the aesthetics of the images we are able to produce and feel that there is a definite correlation between the features of the song and the generated images. With the new craze of NFTs gaining lots of popularity at the moment, we were happy to be able to convert some of our generated images into NFTs and find out more about this new trend.

What we learned

Unfortunately, we learned that picking up and learning some new frameworks on the go, while trying to complete a project in 24 hours with no sleep, is bound to come with difficulties. However we really enjoyed the challenge and learning experience, and in the end, are extremely pleased with the outcome.

What's next for SeeMyFeels

The web app will become a reality, users will be able to upload their favourite songs and hang up posters of their newly generated art pieces in no time ;)