As four Spotify-lovers, music is deeply personal. Whether it’s a study sesh, a drive downtown, or a good time on a Friday night, we’re always searching for the perfect song, artist, or playlist -- the perfect vibe. We created vibecheck as a fun, simple way to bring our conversations and emotions into a custom, curated playlist fit for any moment.
What it does
vibecheck is an application that converts voice call input into a curated music playlist. Based on the conversation, the application generates text and determines the emotion it holds. vibecheck correlates these sentiments to music by relating emotions like happiness, sadness, or “chill” to music features like beat and danceability. Based on the captured “vibe”, the application generates a playlist fit for the occasion or interaction.
How we built it
First, vibecheck uses the Vonage API to collect speech input. The application then transcribes the input using Google Cloud Speech-to-Text and analyzes the sentiment using Google Cloud Natural Language. The sentiments described in the speech input is finally compared to user data via the Spotify Web API. Backend pipelines were joined and involved a combination of node.js and Python to interface with APIs effectively.
Challenges we ran into
We had trouble capturing the recorded input from the phone, but by thoroughly researching the Vonage API and making many test calls, we were able to make it work. Defining appropriate metrics for musical emotions also posed ideological hurdles. Furthermore because of our unconventional user flow, it was difficult to connect the various parts (input, analysis, and output) of the application together. Only through diligent debugging and collaboration did we achieve our goal.
Accomplishments that we’re proud of
We’re proud of pushing ourselves to build something creative that we can see ourselves using!
What we learned
- Web API interfacing
- Google Cloud Speech-to-Text and Natural Language APIs
- Audio Input/Processing with Vonage API and Python
- Github collaboration/project management
What's next for vibecheck
We’re really passionate about continuing work on vibecheck and already have a couple of ideas for feature improvements. First, we want to incorporate more parameters for speech analysis like tone, pitch, pacing, and volume. We’d also love to bring more social features to the app, like comparing users’ favorite artists and automatically building collaborative playlists in Spotify. Finally, we could expand the product to more platforms like Zoom and Discord.
All in all, check your vibe and listen to it, too :)