Inspiration
The inspiration for Sona came from the desire to make music recommendations more personalized and expressive, beyond simple genre or artist preferences. We wanted to create a music recommendation tool that interprets abstract vibes, moods, and themes directly from user prompts. By harnessing the capabilities of AI and natural language processing, we saw the potential to create an app that captures the mood users envision and matches it to music that feels just right for that vibe. Moreover, we saw an opportunity to address a gap in our community of college students: loneliness is an epidemic among young adults in our increasingly digitized society, especially after COVID. With Sona, we wanted to facilitate connections through music.
What it does
Sona takes three one-word prompts from the user to capture the "vibe" they’re looking for in a playlist. Using a pre-trained NLP transformer model, Sona interprets the semantic meaning of each word and matches it to predefined music themes. It then uses the YouTube Data API to generate a playlist of music videos that align with the specified vibe. Users can enjoy music directly on the site without needing to leave the app.
How we built it
We built Sona with a Flask backend and a React frontend. The backend leverages Hugging Face's Transformers library with a pre-trained BERT model to process and interpret user input. By generating embeddings for each prompt, Sona determines which predefined music "themes" match the mood or vibe of the user’s input.
The backend then queries the YouTube Data API to find embeddable music videos that align with this theme, while also incorporating randomness to keep recommendations fresh. The frontend, built in React, takes user inputs, sends them to the backend, and displays the resulting playlist as embedded YouTube videos.
Challenges we ran into
We encountered several challenges in building Sona:
Embedding Compatibility: Some YouTube videos restrict embedding, which initially caused display errors. We addressed this by applying the videoEmbeddable=true parameter in our API query, filtering out videos that couldn’t be displayed within our app.
Limited Variation in Results: Early on, the YouTube API tended to return similar videos, even for vastly different inputs. To enhance diversity, we introduced randomness in query offsets and ran multiple theme-based queries, leading to a more varied playlist selection.
Complexity of Semantic Matching: Interpreting the abstract "vibe" of words required careful refinement of our theme list. We broadened each theme with diverse descriptors to better capture the nuanced meanings and associations of different inputs.
Project Scope and Ideation: We initially envisioned a broad array of ambitious features, including an AI chatbot, facial and emotional analysis, and mood tracking. Through focused discussion and teamwork, we refined our scope, prioritizing core features to ensure a high-quality, polished product.
API: We originally wanted to use Spotify's API for this project, but we realized that it was not fully accessible and open source. As a result, we pivoted to YouTube's API and reformed our backend around that.
Design and Communication: In the early stages, gaps in communication led to minor misunderstandings within both the frontend and backend teams. For instance, our front-end developer initially treated elements on Sona’s home page as static icons, while our UI/UX designer intended them to be interactive input fields. Similar miscommunication on the backend led to multiple overlapping solutions, which required us to reconcile different approaches, impacting collaborative efficiency. Reflecting on the process, we realized that communication between frontend and backend was also lacking. Both teams developed complete solutions independently, which made integration challenging during the final stages. Working in closer alignment—building in parallel and coordinating updates—would have simplified the integration and created a more seamless development process. This experience highlighted the importance of consistent cross-functional communication to avoid misunderstandings and improve project coherence.
Skill Level and Vision Gap: We quickly realized there was a gap between our initial vision and our current skill sets. To bridge this, we focused on adaptability, learning on the job, and supporting each other through tutorials and troubleshooting. We scaled down our project to better match our capabilities, prioritizing quality over quantity in our features.
Accomplishments that we're proud of
As a team with little prior experience with ML, we’re proud to have successfully integrated NLP with music recommendation.
What we learned
Through this project, we learned how powerful AI can be for creative and expressive applications. We explored the challenges of semantic interpretation in NLP and learned how to leverage transformer models to capture nuanced meanings in simple prompts. Additionally, we gained experience in managing and querying external APIs, handling cross-origin requests, and designing frontend-backend interactions.
What's next for Sona: Playlist Generator
In the future, we’d like to:
- Refine the NLP model: Integrate more advanced models like CLIP (which connects images to text), to understand visual or conceptual associations in prompts better.
- Add support for more platforms: Integrate additional music services like Spotify or SoundCloud for audio-only playlists.
- Personalized playlist curation: Introduce user feedback options, allowing Sona to learn user preferences and adapt playlist recommendations over time.
- Expand on themes: Add more theme categories to enhance the diversity and specificity of playlist results, such as themes based on genres, eras, or cultural references.
- Embed the YouTube videos into the playlist to avoid leaving the website.
Log in or sign up for Devpost to join the conversation.