The inspiration for Koala was to address language barriers in personal interactions through technology. When travelling, it can be very difficult to 'merge' with local culture and do simple things like order a meal. We wanted to build something that would eliminate these language barriers and would allow any single person to build a relationship with someone from anywhere in the world.
What it does
Koala allows two users to choose their respective languages and then take turns speaking. As each user speaks, the application translates the spoken word from the first user's language into text of the second user's language. This text is then displayed on the screen. Once it is the second user's turn to speak, the application works similarly and outputs the translated text to another part of the screen. This makes communication quick and easy between two completely foreign parties.
How we built it
Koala was built using a node.js server. The html front-end of the application accepts user interactions as well as speech input. The front-end converts the raw speech data to text in the original language by means of the Google's Cloud Speech API. This text is then sent to the node server where it is translated using the Google Translate API. Finally, the translated text is returned to the user end and displayed on screen. Socket.io is employed to maintain consistent communication between the front-end and the node server.
Challenges we ran into
We faced many challenges in making sure the application was listening for and translating to the correct language at the right time. Getting all of the components to communicate with one another, specifically, implementing the Socket, was very difficult.
Accomplishments that we're proud of
We are proud of setting up the node server and creating a web application which accurately listens for and translates speech.
What we learned
We learned how to build node servers and use the Socket.io framework to send and receive data.
What's next for Koala - A Real-Time Conversation Translator
Further polishing could turn Koala into a hosted web application or mobile application which will make travelling to foreign countries easier than ever before.