Over 136,000 international students choose to study in British Columbia each year, with learning English as their second language. With such a multi-cultural city, being able to communicate in English and different languages is crucial to connecting to others personally.
Despite the language learning apps out there, we realized that many of them do not give personalized feedback on the accuracy of their pronunciation and grammar - which is important to improvement in skills.
We wanted to make practicing languages easier, by creating an online coach (chat bot) that was accessible at anytime, as well as practice in a safe environment without judgment.
What it does
Our website allows learners to choose from a selection of languages to practice on. The chat bot and the user begin a conversation, and the user speaks into their microphone to continue the conversation. The user can then stop the conversation and see an analysis of their speech - such as pronunciation, grammar, hesitation (pausing/filler words) and vocabulary.
How we built it
We used speech-to-text package in npm to convert user dialogues into actual text for grammar analysis. We attempted speech analysis with Microsoft Azure as well.
The method we used to analyze grammar was keeping track of incorrect character counts and display the final grammatical correctness in percentage upon finishing conversation with the chat bot.
Website hosted with Azure
Challenges we ran into
Speech to Text - attempted multiple methods of changing speech to text, experimenting with multiple APIs, such that we were able to convert speech accurately into text into readable files
Speech analysis - understanding the method of grammar analysis to use and comparing it to a baseline
Accomplishments that we're proud of
Experimenting with new technologies Well-flowing user interface Integrating all our work together Working on a project that could benefit many people
What we learned
Without teamwork and supporting each other through the 24 hours, we wouldn't have been able to complete this project. Some seemingly simple concepts can be much more complex than we thought.
What's next for TicTalk
Chat bot speaks aloud Additional metrics for speech analysis Suggestions for conversations with the chat bot Create a social component (and link to social media) to the app, where you can practice with a friend and use the platform as an analysis tool