Inspiration
Karaoke is a favorite pastime and social activity for many people, in fact it's the go-to group activity for a party where I'm from. I use an Echo smart speaker at home to control lighting and make shopping lists. The Echo has a very good microphone and a very good speaker, so naturally, that was the first thing I thought of for implementing this idea.
What it does
Turns any song you request into a karaoke version of the song with vocals removed (using TensorFlow and AI magic), and plays it back so you can sing along!
How we built it
The skill is built using an alexa hosted node.js alexa skill which serves as our front end and basis of interacting with the user. We retrieve the mp3 of the requested song by feeding the request query into a youtube downloader API, and then process it using Spleeter which is an open source tool that uses TensorFlow to remove vocals from songs. These functions all take place on our Gcloud hosted flask backend.
Challenges we ran into
Spleeter and TensorFlow took quite a bit longer than expected to process a song, which is problematic since the allotted response time for an Alexa hosted skill is only 8s. We came up with a temporary workaround by having the user relaunch the skill and request the same song a second time to retrieve the processed song from our backend.
Accomplishments that we're proud of
The result produced by Spleeter is truly impressive, the separated accompaniment is almost as good as the original in most cases. Similar tools we have seen in the past produced much worse results that produced unacceptable levels of noise and artifact.
What we learned
Never underestimate the difficulty of implementing seemingly simple ideas... Sometimes the smallest things can be the biggest headaches Never put off things until the last moment
What's next for KaraEcho - Karaoke Amazon Alexa Skill
Display Lyrics - On our backend, host a dynamic webpage that can be accessed by a smartphone or a connected display, to display the lyrics of the current song playing through alexa
Scoring - Assign a score to a user's singing based on acccuracy in comparison to the original song. Leaderboard/Scoreboard for scorekeeping. Social media integration so friends can share scores and even host competitions among friends
Built With
- amazon-alexa
- amazon-web-services
- aws-lambda
- flask
Log in or sign up for Devpost to join the conversation.