Inspiration
Entering this competition, we initially wanted to make a project about color that could analyze a photo of somebody and give them a color analysis (undertones, complimentary hues, etc). Maxwell on our team was also thinking about using the Spotify API to create a tool, and was curious about how to apply music to more diverse applications. However, combining the two ideas gave birth to a fun project, SpotiPic.
What it does
SpotiPic analyzes an image to find the details about the colors, translating color into audio features, and then finds a song that best fits those features. You can input any .jpg color image into it, and after a short loading period, it displays a beautiful sliding graphic with album art, which when clicked takes you to the song.
How we built it
Firstly, we use React to generate the website and perform front-end tasks such as inputting a picture, sliding the album covers, and displaying the links. Next, this image is passed through a POST API call into a flask backend which runs a Python script that gathers the details of the image. During image processing, we gather the average red, green, and blue (RGB) values of the picture, along with the hue, saturation, and value (HSV) details.
Next, we use a creative set of functions that turn the RGB and HSV values into five audio features: acousticness, danceability, energy, instrumentalness, and valence (happiness). For example, our valence equation can be simplified to "brightness - blue", so that the more blue the image is, the less valence you have, however, it's happier if the blue is bright.
After obtaining these audio features, we use authentication functions that certify us to connect to the Spotify API and a MongoDB server that hosts a dataset of 245 thousand songs and their corresponding audio features. By then parsing through the dataset, we use Numpy arrays to perform vector mathematics to solve for the average percent error of each song's audio features as compared to our generated audio features. We then find the top 5 songs with the minimum amount of error and generate a playlist for each photo. These 5 songs and their details are sent back to the front-end via a JSON conversion, where the corresponding songs are displayed.
HSV and HSL Formatting of Color
Challenges we ran into
- Guido being the absolute GOAT in the Discord
- Elena yapping our heads off smh (Ethan was cool tho)
- Us yapping for 2 hours at the How to Run a Hackathon Workshop
- Too many connections on MongoDB slowing down querying
- json formatting problems
- react/yarn working differently on Mac and Windows (USE DOCKER NEXT TIME)
Accomplishments that we're proud of
Integrating so many tools, most of which were our first time using, to create a project that expands one imagination and gives a different meaning to the world we see around us.
What we learned
React, Yarn, MongoDB, JSON formatting, working with APIs, Project Management, git-hub version control, flask
What's next for SpotiPic
While SpotiPic is used for recreational purposes, a much more advanced version of SpotiPic could be applied in a therapeutic sense for those with visual disabilities. Music transcends words, and being able to understand the world around you through such music can allow for a deeper appreciation and understanding of our surroundings. Furthermore, we would love to dockerize this project so that anybody could import the project on a virtual machine and run it whenever and however they like.
Log in or sign up for Devpost to join the conversation.