Although we are now able to transcribe subtitles for any video in any language, we noticed that there is no technology that facilitates audio-to-audio translation in real time. Oftentimes, when watching a video – such as a foreign movie or TV show – in another language with subtitles, attention is diverted away from the picture onscreen and focused more on the text at the bottom, which can detract from the viewing experience altogether. This audio-to-audio translation therefore allows the viewer to fully immerse themselves in the video they are watching, and concurrently eliminates the need for film/TV companies to rerecord their productions in other languages.
What it does
This android application converts any video that's provided by a user and converts it to a new video with it's audio translated into any language specified by the user.
How we built it
- Microsoft Video Indexer to get a transcript of the provided video and make use of the time stamps
- Android Text to Speech API to convert the audio into any language specified by the user
- Firebase to store the uploaded video files
Challenges we ran into
- Trying to upload a video file using a POST request
Accomplishments that we're proud of
- Since there aren't any APIs to convert speech to speech, we had to first go from Speech to text, and then text to speech.
- Synced speech with the video with right amount of pause. We had to create our own algorithm to do this
What we learned
-Using Microsoft APIs
- Getting better with Android Studio since we all had limited knowledge of it coming into this Hackathon ## What's next for PolyGlot
- Being able to store more videos so that the user does not have to keep uploading the same video to view it again
- Using Bing Speech API so that the user can have the choice of selecting a specific voice that they'd like to hear the audio back from