RealTimeTranslation

Wouldn't it be cool if you could sit down and talk to someone who doesn't speak your language. Well now you can through leveraging googles cloud based translation and speech recognition api.

Comment

Inspiration

By far my biggest inspiration comes from my friends. I have so many friends friends that speak other languages and it makes me incredibly jealous. Because of this, I'm always trying to learn one myself. After years of studying Chinese, Japanese, and Spanish, I've learned one thing. Languages are really hard to learn. So lets cheat. How can I speak a language without knowing it. Ask google apparently!

What it does

In its most basic form my project takes an audio input, i.e a microphone, and it translates that audio into any language you want. The translated audio is then piped into discord for someone else to hear!

How we built it

It wouldn't have been possible without google. I used 3 of their APIs to build it (Speech to text, Text to Speech, Text to Text Language Translation). These APIs with the help of some custom audio recognition python code allowed for seamless and quick audio translation.

Challenges we ran into

The biggest challenge I faced was creating the audio detection code. I had never worked with anything like it before so it was very new to me. I found out that I could take the chunks of audio data being streamed in from my microphone and group the data. If it had a lot of 0's in the chunk, there wasn't a lot of volume i.e (No one was speaking). I used this fact to write an algorithm to detect speech and then start recording the messages. After that I just had to let google take care of the rest.

Accomplishments that we're proud of

I'm genuinely proud that it worked in the first place, but for me I've wanted to do this for years, however, the technology hasn't quite been there yet. Even with my improvements there is a 1-2 second delay in the audio, which is fine for a voice chat, but not for in person yet which is the true goal. I would love to pick up this project again with some more powerful hardware if its ever possible.

What we learned

I learned so much during this project. First and foremost, I've never used Googles' APIs before so that was a learning curve. On top of that, I've never done audio manipulation with python so that was really cool to work on.

What's next for RealTimeTranslation

Since the project has the ability to use any language that google can translate, I would love to set it up to auto-detect the language instead of manually selecting it. I opted out of the practice as it would introduce more latency into the program than I wanted, however, with some improvements, It would definitely be possible.

Share this project:

Updates