As the world gets more interconnected, speaking different languages can be a major barrier to communication. At institutions like the UN, it is not uncommon to see people walking around with earphones in, listening to live translations of each other.

Now think about the refugee crises in Syria or Venezuela. So many people lack good access to medical and legal assistance, because the doctors and lawyers around the world who want to help out often don't speak the languages of the people who need help. If we remove the language barrier from communication, we can give people the power to build community and ultimately bring the world closer together.

What it does

Babel Fish is a video chat app. Whenever someone speaks, that speech is translated in real-time for everyone else in the chat. This lets people who don't speak a common language to easily converse over the internet.

How we built it

We used the Twilio Programmable Video API to set up our video chat. We use the Google Speech-to-Text API to transcribe speech. We wrote our own backend server to forward the transcribed text between clients. We then use the Google Translate API to translate language for each different user.

Challenges we ran into

Video is such a difficult medium to work with, so simply getting video chat up and running was tough. In addition, because this project has so many moving pieces, getting every component from transcription, to video, to translation to all work together was very difficult.

Accomplishments that we're proud of

We are really proud of building a working real-time-translation video chat app, because last night we didn't know if that was even possible.

We can see so many potential applications for this technology. We hope that this kind of technology will be used to break down barriers, and build community around the world.

What we learned

David Learned about JavaScript, and designing user interfaces that look good and work well.

Zak learned about the Twilio Video API, JavaScript and the Google Speech-to-Text API.

Nathaniel learned about Google Compute Engine and the Google Translate API.

Ethan learned about the Google Translate API, Flask, Python, SocketIO and life.

What's next for Babel Fish

We want to build a more scalable app with more features. We also want to add a text-to-speech feature to allow visually impaired users to use Babel Fish, by allowing them to listen to live translations.

Share this project: