Did the Covid-19 pandemic affect your travel plans? Do you miss relaxing on the beach or travelling to a new city? At Summer Jams, you can now bring your travel plans home by listening to music that provides a simulated experience of your favourite travel destination. We created Summer Jams to provide the user with a relaxing getaway during the lockdown.
What it does
Summer Jams brings your pictures to life by audifying them. On uploading a picture, a request is sent to the Google’s Vision API, which detects objects in the image such as birds. These labels are then used with a dataset containing sounds of the corresponding labels. Finally, an audio file is generated accompanying relaxing music and sounds of elements present in the picture. The users can also choose between three different music themes – Acoustic, Piano and Electronic, to adapt the music generated to their tastes.
How we built it
To create the front-end and back-end, we used the Flask micro-framework and web technologies such as HTML, CSS and JS. The website relies on using the Google’s Vision API to find labels in the image. We also used PyDub, a python music library to create the audio files. Finally, we made use of Google Cloud Storage and Google App Engine to deploy the website.
Challenges we ran into
Initially, we struggled with finding pre-existing projects for inspiration that visualise images into its musical elements. Since we didn’t find any, we had to research APIs that would meet the requirements of our application. This often involved reading lots of documentation and testing code, before using them in our project. We also spent a lot of time creating a dataset of sounds and labels, as many pre-existing ones were too large or didn’t meet our requirements.
Accomplishments that we're proud of
Despite the technical challenges, lack of sleep, and occasional procastination, we are very proud to have completed the functionality and the UI design of the website in 48 hours.
What we learned
This hackathon gave us an opportunity to get the experience of using a variety of web technologies and helped us gain confidence in our skills. We learnt how to set up and use the Flask framework from scratch. We also learnt that collaborative problem solving is a very efficient and productive way to come up with solutions. By utilising our strengths, interests and talents we were successfully able to organise the work between the two of us. This greatly improved our progress and helped us become better at time management. Finally, we also learned that taking breaks is important and can help you get a fresh perspective on a problem :)
What's next for Summer Jams Online
In the future, we want to expand the functionality of the website by allowing users to create an account and save their generated music files to playlists. We also hope to create a more extensive dataset of sounds so we can generate a wider variety of music.