-
-
Screenshot
Inspiration
In todays world where everybody is hassling for something and is stressed about a lot of stuff, music has emerged and proved to be a great help. But still human mind can be confusing even for oneself and there are situations in which we know we want to listen to music but don’t know what to listed. Hence, we are developing a machine learning model that can accurately identify facial expressions and recommend music accordingly.
What it does
The Facial Emotion Recognition-based Song Recommendation System is an AI-powered application that analyses a user's facial expressions to detect their emotions and recommends music accordingly. Using deep learning and computer vision, the system captures real-time facial cues, identifies emotions such as happiness, sadness, anger, or surprise, and curates a playlist that best suits the user's mood. This eliminates the need for manual song selection, providing a seamless and personalized listening experience. Whether you need uplifting tunes to boost your spirits or calming melodies to relax, the system ensures that the right music is always just a glance away.
How we built it
We built the Facial Emotion Recognition-based Song Recommendation System using Flask for the backend, which serves as the core framework for handling user interactions and processing facial recognition requests. For face detection, we utilized OpenCV's Haar Cascade Classifier (frontal face haarcascade model) to efficiently detect faces in real-time. To recognize emotions, we trained a deep learning model on the FER-2013 dataset, a widely used dataset for facial emotion recognition, consisting of labeled images for various emotions such as happiness, sadness, anger, and surprise. The trained model processes the detected face, predicts the user’s emotional state, and then recommends a curated playlist that matches their mood. This seamless integration of computer vision, deep learning, and Flask results in a responsive and intelligent song recommendation system.
Challenges we ran into
Accurate Emotion Detection – Training a model on the FER-2013 dataset posed challenges due to noisy and ambiguous labels, making it difficult to achieve high accuracy in real-world scenarios. Some emotions, like fear and surprise, were often misclassified due to their visual similarities.
Lighting and Face Detection Issues – The Haar Cascade model struggled in poor lighting conditions and with occluded faces (e.g., glasses, masks, or extreme facial angles). This affected the accuracy of face detection and, consequently, emotion recognition.
Performance Optimization – Running deep learning inference on facial images in real-time required optimizing the model to ensure fast processing without excessive latency. Balancing model complexity with speed was a key challenge.
Mapping Emotions to Music – Selecting the right songs based on detected emotions wasn’t straightforward. Defining appropriate mood-based song categories and ensuring they aligned with real user preferences required fine-tuning and testing.
Integrating Flask with the Model – Deploying the trained deep learning model using Flask involved challenges like handling large model files, optimizing API response times, and ensuring smooth interaction between the frontend and backend.
User Variability in Expressions – Different people express emotions in unique ways, and some subtle emotions were harder to classify correctly. The system needed to be robust enough to handle a wide range of facial expressions and individual differences.
Accomplishments that we're proud of
Successfully Integrated AI with Music Recommendation – We built a seamless pipeline that detects facial emotions in real time and recommends songs that match the user’s mood, creating a personalized listening experience.
Trained a Deep Learning Model on FER-2013 – Despite the challenges of noisy labels and complex facial expressions, we fine-tuned our emotion recognition model to achieve reliable predictions, making it practical for real-world use.
Optimized Real-Time Emotion Detection – By using OpenCV’s Haar Cascade for efficient face detection and optimizing our deep learning model, we ensured that the system runs smoothly with minimal latency.
Log in or sign up for Devpost to join the conversation.