Inspiration and What it does
We wanted to create something that can be used for entertainment purposes and is fun. The idea of being able to generate a unique video made up completely of gif's closely related to the lyrics your favourite song seemed like the just perfect way!
What it does
Giphyfied generates a unique video that is a made up of gif's closely associated to the lyrics of the song that the user wants to use.
How we built it
We used several python libraries Google Cloud, NLTK, Aeneas, Moviepy and Librosa to convert a song into a gif-video. For this, we had to extract the meaning of the lyrics. We had to strip away the noise in the song which was basically everything except the vocals. Then, we used Google Cloud NLP API to extract the relevant words in each sentence while also generating a map that mapped each line to the exact time that it appears in the song. After this was done, we used Giphy's API to extract about 10-25 relevant gif's related to the key words we selected for each line. Finally, we merged the Gif's into a video by corresponding them to the lyrics' time stamps we had stored earlier.
Challenges we ran into
It was a surprisingly long and tasking process to go from just a song to a gif-video. We ran into several problems during development. Deleting excess noise to extract just the lyrics and recording the time stamps of each line and matching it to the gifs were both long, hard processes as they were both not working properly at first. We had to go over many different libraries to choose which one was the most efficient for our purpose. Combining and storing gifs into short videos which are then concatenated to form the final video was also challenging due to their different frames, fps etc
Accomplishments and What we learned
We learned to use many new libraries to manipulate audio, which was something we had not really done before. We also gained more experience with using Python Flask for frontend, which neither of us had used before. Most of all, we learned how to persevere as the number of times we were almost about to give up out of frustration was way more than all the gif's in this world!
What's next for Giphyfied
We plan to keep working on the project to make it even more accurate in terms of the gif meanings as well as timing in the video generated. This can be expanded in many more fun ways that we are excited to implement soon!
Log in or sign up for Devpost to join the conversation.