Often times we find ourselves listening to a song and wanting to sing along to it. However, there are times when we aren't sure about the lyrics. Finding the lyrics would involve searching for certain keywords and hoping to find the correct song. We created Flyrics to solve this problem. Flyrics uses Artifical Intelligence and Machine Learning to generate lyrics on the fly!
How we built it
Flyrics uses an Artifical Intelligence and a Machine Language approach to generate lyrics on the fly. The project is divided into a backend (python & flask) and a frontend (react native). The user is able to either record a sample of a song, or upload an already existing file in the frontend. The frontend then communicates with the flask server in the backend which is responsible for all of the processing. This processing includes the usage of a python library spleeter which uses a trained model to separate the vocals from the song that was inputted. The vocals are then slowed down in order to increase speech recognition accuracy. The slowed file is passed on to Google's Speech-to-Text API which handles the conversion of the vocals into text. The result is a JSON object which is passed back to the frontend. Finally, the result is displayed in the frontend and the files that were created in the backend are deleted to save disk space.
- We were quite new to AI and ML. This had a bit of a learning curve but was very interesting at the same time.
- We had prior experience in React but React Native was something new to us.
- Getting the speech-to-text API to correctly transcribe the audio was tough as vocals from songs are different than normal speech.
- Very proud of the clean and minimalistic UI for our app.
- We were able to get the desired results and are proud of the end result.
What's next for Flyrics
- Adding the option to be able to save lyrics for future reference.
- Utilizing our existing features to make the app work for other use cases such as transcribing speeches or talks.
- Creating a build-ready mobile app.
- Hosting the backend on a web server.