Speaky is a chatbot for on-the-go foreign language help and language learning.
Users tell Speaky (using text or voice in their home language, or by submitting a photo) what they want to talk about in a foreign language. Speaky responds with a selection of words and phrases in that language.
To help users master this foreign vocabulary, Speaky then creates custom flashcards. These can be reviewed later, or the user can enter a “Quiz” mode that intelligently adapts as the user progresses.
We built two ways to access Speaky: 1) a text service using Twilio, and 2) a Facebook Messenger chatbot to make it easy for the 900 million Messenger users to access the service. We are using IBM Watson’s translator API.
One challenge was that image recognition was not always spot-on. (We are using Microsoft Cognitive Services.) We adapted by suggesting possibilities that the user can select from in their own language before getting the translation.