Both of us love watching basketball games. Very often when we want to catch up with the games when traveling on the road, we can't watch the livestream due to limited data usage. So why can't we have a live text update of the games we love? Even though there are websites and mobile applications that tell you the score and describes the game progress briefly in real time, we want more. As we know, commentators act as a great stimulus to the audience's emotions, what if we can automate the process and get to know what commentators are saying in real time.
What it does
Our project takes in an audio input from commentators in the game and transcribes it into text for users to get live text updates of the game. For non-native speakers, they can use our application as well as we support the real-time translation of the commentaries.
How we built it
We use Google Cloud Speech API to transcribe and translate the audio pieces stored on Google Cloud itself. By hosting a simple flask server, we are able to establish the communication between the backend and the frontend, enabling user to choose what audio files that they want to test and to see the processed output.
Challenges we ran into
At first, we took really long to come up with an idea that interest us both, easy to build and useful.
Then we spent a lot of time trying to use Azure Speech services as we are really overwhelmed by the accessibility and the convenience it is able to provide. However, we have found out that its API currently only allows for limited access since it's still in the preview phase and requesting access may take too long. Therefore we change our focus to using Google Services instead.
Another challenge we face is that this hackathon is filled with so many interesting games and talks that we have a hard time resisting the temptation.
Accomplishments that we're proud of
We finished it on time!
What we learned
- It is important to come up with an idea before the hackathon starts, and it can save us LOTS OF TIME.
- Try new things quickly in a short-time event if the old ideas fail.
What's next for NBA Commentary Speech to Text
- A more user-friendly UI
- Use real-time voice input instead of recorded sound clips
- Pre-process the input sound like decreasing the background sound volume to increase the accuracy of transcribing commentaries