Pulse

AI-powered music app that detects your facial emotions in real time, matches your mood to Spotify tracks, and features an AI DJ that reacts and speaks as your vibe changes.

Comment

Pulse 🎧 — Emotion-Aware AI Music Experience

Inspiration

Music has always been closely tied to human emotion. Most people already choose different playlists depending on whether they are studying, relaxing, stressed, working out, or feeling emotional. We wanted to explore a simple question:

“What if music could automatically adapt to how you feel in real time?”

That idea became Pulse — an AI-powered emotional music companion that detects facial emotions through a webcam and dynamically matches the user’s mood to Spotify music recommendations. We also wanted the experience to feel alive and interactive, which led to the addition of an AI DJ persona that verbally reacts to emotional changes like a real intelligent assistant.

Rather than creating another static music player, our goal was to build an experience that felt futuristic, adaptive, and human.

What It Does

Pulse uses real-time facial emotion detection to identify emotions such as:

  • Happy
  • Sad
  • Neutral
  • Angry
  • Fearful
  • Surprised

Once a mood is detected and stabilized, Pulse searches Spotify for matching music and updates the playlist dynamically.

To make the experience more immersive, an AI DJ assistant speaks to the user using AI-generated voice responses such as:

“You seem stressed. Switching to a calming playlist.”

The result is an adaptive emotional music experience that feels intelligent and interactive.

How We Built It

Frontend

We built the frontend using:

  • React
  • Vite
  • Tailwind CSS
  • Framer Motion

The interface was designed with a modern brutalist-inspired aesthetic to create a sleek and immersive user experience.

Emotion Detection

For emotion recognition, we used:

  • face-api.js

The webcam feed is continuously analyzed to detect emotional confidence scores in real time.

To avoid chaotic playlist switching caused by noisy predictions, we implemented:

  • emotion stability checks
  • confidence thresholds
  • cooldown timers

This allowed Pulse to feel more natural and less reactive to micro-expressions.

Spotify Integration

We integrated the Spotify Web API using:

  • Spotify OAuth Authentication
  • Track search APIs
  • Playlist recommendation logic

The app maps emotions to different music categories such as:

Emotion Music Style
Happy Upbeat / Dance
Sad Piano / Acoustic
Angry Calm Lo-fi
Neutral Focus / Chill

We also introduced randomized search logic to avoid repetitive track recommendations.

AI DJ Persona

One of the most fun features was the AI DJ system.

We combined:

  • Google Gemini TTS
  • emotion analysis
  • dynamic prompts

to create a voice assistant that reacts emotionally to the user’s state.

This transformed the project from:

“AI mood music player”

into:

“An intelligent emotional audio companion.”

Challenges We Faced

1. Spotify Preview Limitations

One major challenge was that many Spotify tracks no longer provide public preview URLs due to regional and licensing restrictions.

This meant we could not reliably auto-play official previews for every track.

To overcome this, we redesigned the experience around:

  • dynamic recommendations
  • Spotify linking
  • adaptive playback logic

instead of relying entirely on previews.

2. Emotion Detection Instability

Facial emotion detection changes rapidly frame-by-frame.

Initially, the music changed far too frequently and created a chaotic experience.

We solved this by implementing:

  • emotion smoothing
  • stability timers
  • cooldown systems
  • confidence filtering

This significantly improved the realism of the AI behavior.

3. Recommendation Repetition

Spotify search results frequently returned the same top tracks repeatedly.

We improved the recommendation engine by:

  • randomizing search queries
  • rotating mood keywords
  • tracking recently played songs
  • avoiding recent duplicates

This made the music feel more dynamic and personalized.

What We Learned

Throughout this project, we learned:

  • how emotion recognition systems behave in real-world conditions
  • how to integrate OAuth securely
  • how to handle API rate limiting and unstable external APIs
  • how to design adaptive user experiences
  • how important UX and presentation are in AI products

Most importantly, we learned that building AI products is not just about machine learning accuracy — it is also about creating an engaging and believable user experience.

Future Improvements

In the future, we would love to expand Pulse with:

  • voice emotion detection
  • typing pattern analysis
  • smartwatch heart-rate integration
  • productivity mode
  • multiplayer group mood syncing
  • fully AI-generated music

We envision Pulse evolving into a complete AI emotional companion for productivity, relaxation, and entertainment.

Final Thoughts

Pulse started as a simple idea:

“Can AI understand your mood through music?”

But during development, it became something much more interactive and expressive.

By combining emotion recognition, adaptive music recommendation, and conversational AI behavior, we wanted to create an experience that feels like the future of personalized entertainment.

Built With

Share this project:

Updates