The Emotional Oranges

Log in page
Upload Page
Playlist Page
Playlist Page

Inspiration

The main inspiration behind this idea was the love of music and the impact that it creates on everyone. Every single piece of music invokes a sense of feeling and oftentimes we desire the perfect song for the mood we are feeling. Creating a web app that allowed everyone to experience the perfect song at their desired time, seemed like a worthwhile and incredible venture.

What it does

The web app gets asks a user to login to their Spotify account and upload a picture of themselves. Our trained tensorflow model will detect the mood of the image and would output either a happy, angry, disgusted, fearful, sad, neutral, or surprised mood. A playlist with 16 songs will then be outputted to match that mood. The moods of the songs are found using a combination of their valence and energy metrics. The population of songs were collected from the user's favorite songs on Spotify when they logged in.

How we built it

We first needed to create and train the ML model that uses image classification to recognized the moods of images. We found the training dataset we needed on Kaggle and then utilized Python and the Tensorflow library to successfully train the model. We also needed to learn how to use the Spotify api and how to gather information about songs from it. We then used React.js to create the web app. This included creating a login page that interacted with Spotify, an upload page, and a playlist output page. the web app worked with a Firebase storage and we needed to ensure that this connection worked smoothly. Finally, the web app was styled using Chakra UI.

Challenges we ran into

A challenge we ran into is the fact that training the model on the giant database of image took nearly 30 minutes. However, as the model was incorrectly saved the first time, this process needed to be repeated. Another challenge we ran into is working with the Spotify api. There were issues with gathering our desired data from Spotify that required a lot of time remedy. Afterwards, we had issues working with getting an output from the model when we requested this on the web app, but were able to resolve this after a bit of experimentation.

Accomplishments that we're proud of

The app successfully accomplishes what we set out for! Any image can be uploaded to the web app and a playlist fitting the mood will be presented. Furthermore, the web app looks nice and orange!

What we learned

We learned a lot about ML models and to successfully create and utilize them. We also learned more about React.js and how to use it effectively. We learned more about working with apis and how to patiently fix issues that we might face.