Inspiration
We drew our inspiration for this project from the well-known adage of “A picture is worth a thousand words.” Authors and poets are masters at conveying vivid, precise images with only a few words. One famous example is The Red Wheelbarrow by William Carlos Williams. With only 16 words, Williams’ poem is the subject of countless lengthy essays and artwork. We know words can inspire art. We also know — so can music.
Many artists have credited music as a muse for their work. Just as music can inspire art, art can inspire music. Van Gogh’s Starry Night painting, for example, inspired Henri Dutilleux’s orchestral work Timbre, Space, Movement. In French, the work is subtitled "The Starry Night" after this painting. Together, words and art and music have the power to transport us to faraway places, share culture, and evoke deep primal feelings at the core.
Given almost a year of quarantining due to the COVID-19 pandemic, there are things we miss like being able to go relax on the beach or take a group photo where everyone’s smiling. So how can we brighten up our lives, bring travel destinations home and the feeling of hanging out with friends and family, and encourage a different frame of mind?
We do this by curating a Spotify playlist that provides a simulated experience through listening to music . See what songs could be inspired by your photos. And let our app help you immerse in how you feel — through images.
What it does
A user would first upload an image.
It gets processed through algorithms that extract keywords used to calibrate the audio features* in a song track that are added to the Spotify playlist.
So in a nutshell, our web app converts images into a Spotify song playlist.
*Audio features, for example, include valence for the musical positivity conveyed by a track, or danceability, which describes how suitable a track is for dancing based on a combination of musical elements including tempo, rhythm stability, beat strength, and overall regularity.
How we built it
We used HTML, JavaScript, and CSS for the front end, and Python Django for the back end of the website. Python libraries like Everypixel, NLTK, Colorthief, and Algorithmia were used to convert an image into keywords and audio features that could be used to select songs from Spotify.
Challenges we ran into
The biggest gap in this project was in transforming objective keywords from images to subjective 'moods' that define songs. We also had to figure out how to keep our client ID's and client secrets confidential. Additionally, a number of features in the front end were eventually taken out and images were not output due to prioritizing back end configuration.
Accomplishments that we're proud of
Just the fact that it's possible for a machine to determine the emotion of an image is awesome. We accomplished the goals we had set out to achieve for this hack, and getting to the end left us with an incredible and fulfilling feeling.
What We learned
David
A whole lot about integrating with Spotify API
Anita
Given limited exposure to coding pages in Django and the number of design questions to consider, I considered how to convey this project idea in a simple way while keeping the project clean and to the point with its idea.
Ryan
I've never used online APIs before, and I was surprised at how easily we leverage the incredibly complicated algorithms to our whims.
Prannav
I learnt how to use different APIs to get specific AI tasks done. I also learnt about .env files and some basics of Django.
What's next for Image2Audio
Clean up the front end
More refined and personalized algorithms to determine playlist!
Log in or sign up for Devpost to join the conversation.