Spotifeels

Inspiration

While it is already one of the best in the market, the Spotify playlist generator lacks the ability to adapt to it's surroundings. Users often blindly trust Spotify's ability to recommend songs to deal with their current mood, and the engine lacks the ability to read that well. This is why we integrated Hume's facial emotion detection tool to adjust the Spotify recommendations to fit the user's current mood.

What it does

This app simultaneously opens 2 windows, a Spotify playlist generator, and a webcam running in the background. The playlist generator allows the user to log into their account, choose a song to generate similar recommendations and do a basic adjustment of the vibes of their recommendation. The webcam reads the user’s face, providing a stream of emotional scores, such as happiness, excitement, angst, etc. This is reflected in the playlist generator, which refreshes every 5 seconds to adapt to the users’s current emotions.

How we built it

We wanted a way to collect real-time data about the user’s emotional scores and thus their opinion on the current song, so we decided to incorporate Hume AI’s streaming API to get facial expression data from the webcam. We combined this with a Spotify playlist generator in order to generate a more specific and accurate list of song recommendations than possible with just Spotify’s base recommendation system.

Challenges we ran into

Given the complex nature of the Hume video detection API, the main challenge was reading the source code to a point where we could make API calls between Hume’s programs and our base web app. These synchronizations are tricky and difficult given our level of experience working with large-scale web applications

Accomplishments that we're proud of

This was our entire team’s first hackathon experience, and we are all proud of the fact that we were able to produce a complete project. Additionally, we are proud of taking upon a more difficult task, integrating AI into our very first project and succeeding, reading through the dense code to connect it to our own idea. We believe that the project we built and the idea we developed have the potential to be scaled into the future as well, and hope to do so.

What we learned

Much of our potholes came from not reaching out to companies and mentors for help more, trying to decrypt all the information and code we decided to use. Much of our largest problems – especially in networking – could be streamlined in a fraction of the time by consulting experienced developers, leaving more time for our executing our creative visions rather than debugging rudimentary problems.

What's next for v

We could see this project developing into a Spotify extension that runs in the background, and hopes to add features to allow for continuous smart recommendations added to the song queue, which would mean seamless, personalized music with no need for the extra steps of creating playlists or running a program each time.

Built With

• javascript
• webapi