Vibe Check

Inspiration

There's always that one friend that has terrible music taste but loves to hog the aux.....

What it does

Vibe Check reads the social situation of the crowd through the input from the webcam and mic in order to compute a real-time "hype score" from crowd motion, emotion, and loudness, then dynamically fades throughout our personally curated Spotify playlists and lighting to match the energy.

How we built it

We built a two-threaded OpenCV pipeline for real-time video capture and processing, layering YOLO for people detection and DeepFace for emotion recognition on top of the raw frame stream. That feeds into a FastAPI backend which pushes live hype score updates over WebSockets to a React dashboard and triggers Spotify and Philips Hue APIs on every mood change.

Challenges we ran into

Our biggest challenge was optimizing the frame rate of the webcam pipeline as the computer vision model was fetching and processing frames too aggressively, causing serious performance degradation until we decoupled capture and inference into separate threads. Accurately reading crowd mood through facial expressions was also harder than expected, requiring additional fine-tuning before the model could reliably detect subtle shifts in energy.

Accomplishments that we're proud of

A major success was getting the dj-brain/logic to work as we were constantly refactoring the dj-brain algorithm, ensuring that it was properly adjusting the music based on the input through the webcam and mic. It essentially decided which mood level was appropriate for the crowd.

What we learned

This project allowed us to gain direct hands-on experience with working with computer vision and machine learning. It's our first time working with machine learning like this so we look forward to future projects that are to come incorporating ai/ml.

What's next for Vibe Check

We want to train the model to build crowd profiles based on listening history and real-time feedback, so the system gets smarter about a specific audience over time. That would require a persistent database layer and more sophisticated model training, but the foundation is already there.