VibeS

• 

  Initial Screen

• 

  Loading Screen

• 

  Result Screen

Inspiration

• While at the CalHacks venue, a team member, Jeffrey, took a picture of the city buildings and wondered, "How can we capture the vibe of this picture?". The team had previously wanted to work on an AI and music-related project, so we got to work in trying to successfully capture the VibeS.

What it does

Provided a picture and a Spotify account, the application will provide a curated playlist that captures the VibeS of the picture based on the user's preferences.

How we built it

We trained a visual transformer BEIT from Hugging Face using Intel's Cloud Computing services that categorizes pictures. Using some of these categories, we then fetch from a Convex database populated with the user's songs from his "niche" playlists and provide the top results.

Challenges we ran into

Making a dataset for the visual transformer was challenging as well as learning new technologies like Convex database mutation and querying.

Accomplishments that we're proud of

We are proud of building accurate models for the time of the day and environment classifications. We are also proud of being able to build an appealing front end for our project.

What we learned

We learned a lot of skills such as training and testing models using Hugging Face and Intel's Cloud Computing Services, using Convex for database mutations and queries, fetching user information using Spotify's API, and getting more comfortable building in TypeScript and React.

What's next for VibeS

More data, more classifications for images, and more features such as instantly adding a playlist to Spotify, linking results to Spotify song links, etc.

Built With

• beit
• cloud-computing
• convex
• hugging-face
• intel
• react
• spotify
• typescript