On our team, we are big music enthusiasts and like many others, have all experienced the annoying feeling of finding a song that we really like in a different language, but not being able to understand the meaning behind the lyrics. It is one thing to find a translation online, but something completely different to be able to experience the music along with the words, in a language that you do understand.
What it does
Our web app takes either the title of a song, or a .wav audio file and gets the lyrics of the song, through a search engine that we customized. We then use the googletrans API to translate the song into the user's language of choice and use a text-to-speech API that then converts the lyrics to spoken word. The pitch and frequency of the words are then manipulated to match those in the original song. Finally, we overlay the music and end up with the same song, but in a different language.
How I built it
The project was coded in python and HTML5 and made use of a lot of web APIs. We first setup the project with python and worked on basic functionalities such as: making API calls simultaneously to get lyrics from from the song name/.wav audio file, to translate lyrics from one language to another, to convert translated lyrics into speech (mainly using google api), and to get audio analysis data of the music and tagging the pitches back on the translated speech. Then, we combined all the functionalities together to return the song in a different language and play it on browser.
Challenges I ran into
We had some issues when trying to alter the pitch of the spoken word. There was LOTS of debugging involved!
Accomplishments that I'm proud of
We are really proud of the fact that we took a risk in choosing a project that was quite ambitious and that is very different from any other ones that we have done in the past. We worked very well as a team and learned so many new things together. Additionally, 2 out of the 4 of us haven't ever coded in python before, but were able to learn on the spot and make very helpful contributions to the project.
What I learned
We all learned a lot more about queuing web API requests and manipulating soundwave data. Although, no one on our team has any experience with UI, we were still able to create some sort of a web application, that interacts with the user and takes their input for song titles and language requests. For the two of us who have no experience with python, it was a very cool challenge to figure out how to both accomplish the required tasks, all while using new syntax.
What's next for lango-tune
The are still some challenges with pitch-matching that need to be resolved. Some improvements could also be made to the music overlay. As a team, we wish to continue working on this project after the hackathon is over, since it is something that we are all genuinely interested in and passionate about!