34 - TSPeach

Communication is everything! For non-native English speakers, it can be difficult and even embarrassing not being able to pronounce certain words. Using Assembly AI we aim to solve this!

Comment

Inspiration

We tried to figure out what kept us connected during the pandemic other then the non ending zoom meetings or the occasional time you spend in class together, and fundamentally this all came down to our ability to just speak and once we started thinking about it we couldn't stop

What it does

We created a web app that displays a sentence that the user can read and using assembly ai’s real time word detection API we stream what the user is reading, while providing feedback on their correctness. Using a Profanity free comprehensive dictionary, we randomized which words are shown to the user to help make each sentence challenging in a different way.

How we built it

In our design process, we started with the idea. After coming up with our idea we started our research to find the best way to implement the features we wanted to use and after realizing we had access to assembly ai, we knew it was a match made in heaven. Afterwards, we started designing basic functionalities and creating flowcharts to identify possible points of difficulty. After our design process, we started developing our project using html, css, NodeJS, JQuery and assembly ai.

Challenges we ran into

We initially hoped to use python as our main language, however, learning Django while also finding ways to provide accurate feedback proved to be too difficult within the time frame which further lead us into building our project in JS. Furthermore, learning Node.js and assembly ai was also significantly difficult considering the time frame.

Accomplishments that we're proud of

Having ran into countless problems with Django Web Kit and Python in the beginning, we decided to switch to a JavaScript base. Now, with only half the remaining time left, we were forced to be creative and work diligently to finish before the deadline. Ultimately, the end product was better than we could have hoped for, and incorporated many completely new concepts to us. It was this ability to problem solve and learn quickly that we are both very proud of ourselves for.

What we learned

Along the way to finishing our project, some of (far from all) the things we learnt about were: web device interfaces for recording audio, networking and websockets to help communicate with external APIs, audio streams with machine learning, running javascript as a backend, using NodeJS modules, hosting client and server side platforms, and in general, user experience optimization as a whole.

What's next for TSPeach

We hoped but were unable to include was the increasing of user feedback based on their pronunciations. We initially wanted to analyze and compare each user's pronunciation to a text-to-speech engine, however it was too hard to do in the time frame, so this would be another feature we would love to add. Optimizing our interface with Assembly AI would be our next major goal. Currently, the asynchronous approach to handling responses from Assembly AI uses a single async thread, however having multiple collaborating would be the ultimate goal.

Built With

Share this project:

Updates