Project Resonate

Inspiration The inspiration for Project Resonate stemmed from a shared frustration with existing music recommendation systems that often fall short in reflecting our current emotional state. We realized that relying solely on listening history or generic algorithms ignores the powerful, real-time connection between music and emotion. We envisioned a system that could intelligently understand a user's mood and offer truly personalized music that resonates with their feelings in the moment, enhancing well-being and discovery. What it does Project Resonate is an AI-powered music recommendation system designed to curate playlists based on a user's real-time emotional state. It utilizes computer vision to analyze facial expressions detected via a webcam, identifying emotions such as happiness, sadness, anger, or calmness. The system then generates a personalized playlist from a music database that aligns with the identified mood. Users can initiate the process simply by starting the music application and allowing camera access. The application can also dynamically adjust recommendations as the user's mood changes, providing an adaptive and engaging listening experience. How we built it We built Project Resonate using a combination of technologies for emotion detection and music recommendation. Emotion Detection: We utilized Haar Cascade to detect faces in real-time video frames captured by a webcam. For emotion classification, we employed a Convolutional Neural Network (CNN) trained on a subset of the FER2013 dataset, focusing on happy, sad, angry, and calm expressions. The CNN architecture included convolutional layers to extract features, max-pooling for downsampling, and dropout layers to prevent overfitting. Music Recommendation: We used a Spotify song dataset, extracted using the Spotify Web API. We applied Principal Component Analysis (PCA) for dimensionality reduction and K-means clustering to group songs into four distinct mood categories based on features like valence and energy. The clusters were mapped to emotional states using the circumplex model of affect. Content-based filtering is used to refine the recommendations within the detected mood cluster after a user selects a preferred song from a preliminary list. The system then plays the top recommendation using the Spotify API. User Interface: A web application, potentially built using Streamlit. Challenges we ran into Developing Project Resonate presented several key challenges: Real-time Emotion Detection Accuracy: Ensuring the accuracy of real-time emotion detection, especially under varying lighting conditions and user expressions, proved challenging. There's inherent subjectivity in how humans express emotions and how models interpret them. Dataset Limitations: The availability of standardized and diverse datasets for both facial expression recognition and music mood labeling can be a hurdle. Smaller datasets can lead to issues like overfitting during model training. Computational Demands: Training and deploying robust deep learning models, particularly for CNNs and reinforcement learning components, required significant computational resources. Nuanced User Preferences: Accurately reflecting subtle user preferences within a given mood, based on limited features, was a complex task. Addressing the "cold start" problem for new users or songs also required careful consideration. Privacy Concerns: Collecting sensitive data like facial expressions necessitates careful handling of privacy and security concerns. Accomplishments that we're proud of Despite the challenges, we achieved several accomplishments we're proud of: Functional Prototype: We successfully developed a working prototype that demonstrates real-time emotion detection and delivers mood-based music recommendations. High User Satisfaction: Early user feedback has been positive, indicating that the system effectively provides music recommendations relevant to their emotional state. User satisfaction was reported at an average score of 4.2 out of 5 in one evaluation. Efficient AI Integration: The project showcases the practical application of AI and machine learning techniques for personalized music experiences. The CNN achieved a test set accuracy of about 70.28% in our tests. Enhanced Discovery: The system has demonstrated the ability to suggest music across traditional genre boundaries, based on underlying emotional characteristics. Adaptive Learning: The reinforcement learning components showed continuous improvement in recommendation quality over time, adapting to user preferences. What we learned Developing Project Resonate provided valuable insights: Importance of User Experience: While backend processes are crucial, a seamless and intuitive front-end is essential for user adoption and satisfaction. Power of Emotion-Based Systems: Integrating emotional intelligence into recommendation systems creates a more meaningful and impactful user experience, moving beyond mere content consumption. Data and Model Management: Effective handling of diverse datasets and managing computational resources are critical for developing robust and scalable AI solutions. Nuance in Recommendations: Developing systems that capture the nuanced relationship between music and emotion requires careful consideration of feature selection and algorithm design. Ethical Considerations: Ensuring data privacy and security when working with sensitive user information is paramount in building trust and promoting responsible AI development. What's next for Project Resonate Looking ahead, we envision several exciting next steps for Project Resonate: Improved Emotion Recognition: We plan to enhance the accuracy and robustness of the emotion detection module by exploring advanced deep learning models, integrating multimodal data (like voice analysis), and expanding the range of recognized emotions. Enhanced Personalization: Future enhancements will focus on deeper personalization, such as incorporating user listening history, contextual factors (time of day or activity), and incorporating explicit user feedback to further refine recommendations. User Interface Refinement: We aim to develop a more user-friendly and aesthetically pleasing interface, potentially exploring mobile app development for wider accessibility. Cross-Platform Integration: Exploring partnerships with streaming services and expanding cross-platform compatibility will allow for richer music libraries and broader reach. Advanced Recommender Systems: We plan to investigate advanced deep learning-based recommender systems, potentially leveraging techniques like deep Q-learning, to provide more sophisticated and context-aware suggestions.

Built With

• and-scripting.-javascript:-for-any-front-end-functionality-that-runs-in-the-browser
• data-processing
• javascript
• pytho