Inspiration
Many language schools are closed during quarantine. Some classes moved online, but some are permanently unavailable. One default of online language learning is that real-time interaction is no longer as efficient as it is in person. Therefore, I wanted to make use of Microsoft Azure Speech SDK to build a self-learning and self-correcting language learning platform that has almost simultaneous response from the computer.
What it does
The application randomly selects a movie line from Cornell Movie Dialogs Corpus database, and let user record their speech from the browser. With the help of Microsoft Azure Speech SDK, the backend analyzes the wav file and returns assessment results to the user, including sentence accuracy, pronunciation, completeness and fluency. It also gives pronunciation accuracy for each word. With these information, the bowser displays the line with different colors of each word so that user knows which word they need to pay attention with.
How I built it
Cornell Movie Dialogs Corpus (database): 304,713 utterances from 617 movies
Flask, Microsoft Azure Speech SDK (backend): Extracts movie lines and perform assessment. Creates APIs for frontend usage.
React, axio (frontend): Displays the assessment results
Challenges I ran into
It was my first time using Microsoft Azure and Flask, and I worked solo. I had to start from scratch, and I ran into countless issues, not surprisingly, though they were finally resolved.
Accomplishments that I am proud of
This app is useful! I can use it to practice my spoken English!
What's next for Fluently
- Next, I will add a login form and a database so that users can store their historic data in the database for further analysis. Depending on the accuracy, fluency, completeness, we can qualify how much improvement a learner has made.
- Also, I will add a upload feature for users uploading their own database so the learning material won't just limit to movie scripts.
Log in or sign up for Devpost to join the conversation.