AI Emotion EQ

Using AI facial recognition and Physiological Data from a FitBit to infer emotion and play songs enforcing that emotion through a Spotify integrated web portal.

Comment 4

Inspiration

Spotify's DJ feature lacks customization, but having music that aligns with your mood has been proven to enhance enjoyment and even improve exercise performance. So, why not create something that integrates with the world's largest music platform to provide a more personalized experience?

What It Does

We assign tempo and valence scores to your top Spotify songs. Using your heart rate and facial expressions, we play music and visuals tailored to your emotional state in real time.

How We Built It

The Website

The website is built with a Vue.js frontend and a Laravel PHP backend. It’s designed to aesthetically adapt based on emotional and physical measures.

The Server

The server leverages the Laravel framework to act as an intermediary between the website and the measures. It facilitates API calls for storing and retrieving data seamlessly.

The Measures

  1. Facial Tracking: Emotional facial tracking is enabled by a pre-trained model found at [https://github.com/ArnabKumarRoy02/ResEmoteNet]. This model assigns numerical values to emotions using live camera input.
  2. Heart Rate Monitoring: Heartbeat data is captured live through the Fitbit API integration.

Challenges We Faced

  • Spotify API Limitations: Spotify deprecated key functions for generating song recommendations and retrieving valence and tempo data. To innovate, we utilized the ChatGPT API to approximate these values.
  • Accuracy in Real-Time Face Tracking: The pre-trained facial tracking model needed refinement to enhance real-time accuracy.

Accomplishments We're Proud Of

  • Designing an aesthetically pleasing and functional website.
  • Completing a technically challenging full-stack project within a tight timeframe.

What We Learned

  • How to work with complex APIs.
  • Gaining hands-on experience with AI models.
  • Developing and integrating APIs efficiently.

What's Next for AI Emotion EQ

  • Developing a mobile app version for greater accessibility.
  • Further refining AI models to improve accuracy and responsiveness.
Share this project:

Updates