SnowProblem is a conversation tool that will work in any of 60+ languages. Whether learning a language, traveling, or you just want a question answered in your native tongue, we have you covered. To make the journey smooth, the user is accompanied by two cute winter friends, Snowy the polar bear and Slushy the penguin.
When studying a language, one of the most important parts of the process is having conversations with friends and classmates in the desired language. Unfortunately, most learners don't have access to these resources frequently, if ever. We want to make sure that this is no longer true, that it is Snow Problem to converse and really learn how to use your new language.
Our stack runs using a pairing of numerous services elegantly strung together. To make it work, quite a few hacks were used. Firstly, the app is developed namely as a chrome extension. These don't have access to voice input of any kind, so js injection, object tracking, and a few clever tricks combine to form a 4 layer deep process to finally get the parsed response in any language. From there, it will determine if it must be translated to the base language of the underlying chat application, translate using Google translate, collect an in context response from the chatbot, and translate it back to the expected language. Finally, it uses TTS to produce the desired language verbally. After waiting for itself to complete speaking, it will once again await the user.
Our second feature is for when you are dropped in a real environment without sufficient language skills. Users can select to languages to be used, speak into their device, and then the other party will hear their native language. When they want to switch speakers, they simply click on the microphone nose of our adorable winter friends, and the other party will instantly be able to continue the conversation. This will eliminate much of the difficulties encountered while travelling and interacting with a wide variety of individuals.
To push further, we even made a generic web app that runs in any browser with supported components, including mobile phones. This extends the reach far beyond a desktop and makes the features users need available wherever they are.