Inspiration
Music has a profound impact on human emotions, serving as a universal language that transcends cultural and linguistic barriers. It has the power to evoke a wide range of feelings, from joy and excitement to introspection and melancholy. Whether through the rhythmic beats that induce movement or the emotive melodies that tug at our heartstrings, music has the ability to shape and enhance our emotional experiences. We wanted to explore how we could enhance these emotions and help productivity.
In recent years, the adoption of smart home technology has witnessed an unprecedented surge. As this trend continues to gain momentum, the integration of innovative applications like Spotivibe can turn traditional homes into intelligent, interconnected spaces. Spotivibe, a cutting-edge application, takes the smart home experience to new heights by seamlessly integrating emotion detection technology with personalized music recommendations.
What it does
Enter SpotiVIBE, the playlist that enhances your emotion in your day to day life by capturing your current motion over each song and queues the next song. SpotiVIBE is trained using machine learning to detect 6 different human emotions (angry, excited, happy, neutral, sad, tired.) The computer vision also enables hands off features for Spotifiy, such as skipping and pausing the current song using hand emotions.
How we built it
When developing the frontend we mainly implemented React and Spotify API When developing the backend we incorporated python, tensorflow, opencv, numpy, pillow, face_recognition, Flask, SocketIO, base64.
We curated our own database full of pictures of various emotions from hackers and celebrity face databases into the 6 categories. We trained a tensorflow model to detect these emotions. The frontend receives the emotion and uses the Spotify API to play song based on the users previous listening history.
Challenges we ran into
Being our first time working with computer vision it was difficult to link the webcam data from our frontend to our backend. Processing the images proved challenging due to the time constraints related to creating a comprehensive database of emotions to train our ML model.
Accomplishments that we're proud of
The ML model was able to distinguish between the different emotions.
What's next for SpotiVIBE
As an extension we could have implemented are different settings. For example if you click on study mode and the model detects tired, then instead of playing sleepy music it could play your study mix. We also want to cater to the different uses of SpotiVIBe in the future, such as using it while driving or playing specific music while streaming.
Log in or sign up for Devpost to join the conversation.