IPA+

Chrome extension for aiding pronunciation based on the International Phonetic Alphabet. Immediately transcribe web text into IPA and get animation of basic mouth positions.

Comment

Inspiration

The Language Barrier is a huge factor that determines the efficacy of communication in a new language, not to mention the mental health impact and social exclusion that comes with not being able to effectively communicate in a foreign language. We hoped to create a Google Chrome extension that can leverage the power of the International Phonetic Alphabet (IPA), which can be learned quickly and easily in one’s native tongue, in order to assist in pronunciation. Moreover, we wanted to assist specifically in pronunciation with a visual guide along with tips to help one pronounce words and phrases on the fly, using the internet as a reading guide. We believe this can help individuals hoping to improve their pronunciation in a new language and those undergoing speech therapy.

What it does

This app is a Google Chrome extension that will take any English text found on any online webpage and translate it into the International Phonetic Alphabet, coupled with an animation! The goal is to make language learning easy, with built in diagrams to tell you exactly how to pronounce each syllable. Ideally, it would include text to speech examples to accompany the translation, but unfortunately this was not implemented at treehacks.

How we built it

The IPA+ plugin was built with a statically-built React frontend that controlled the logic for storing highlights and making API calls for translating English to IPA. We used Python scripts to facilitate the translation and hosted the platform on a Django server and database. Once translated, the resulting output is rendered with mouth animations for each syllable. IPA+ is designed with CLIP Studio Paint Pro for graphic design/animation.

Challenges we ran into

None of us were experienced in developing extensions for Chrome prior to this, so many of our issues revolved around interpreting the Chrome documentation. When it came to voice recognition, we struggled to implement a handful of speech-to-text and text-to-speech APIs, joining the front end with the back, and determining the best visuals to match the sounds. Moreover, this was our first time working with Soundify and SoundHound, which we attempted to learn from scratch during this project.

Accomplishments that we're proud of

We created a new and intuitive image set in order to assist people in developing proper pronunciation.

What we learned

We learned the benefits of IPA and the use of images to assist in speech and pronunciation. With regards to coding our project, we learned deploying React apps, making chrome extensions, making Django API calls efficient, node packaging and more!

What's next for IPA+

We would like to implement text-to-speech voice clip examples! Additionally, in an ideal world, we would be able to take voice clips and tell you exactly where your pronunciation could improve compared to the accepted IPA standard. We understand not everyone is able to immediately understand IPA, and we hope to broaden its comprehension, so we would like to work to make this more accessible to all and more user-friendly. Eventually, we would also like to expand to include a handful of languages outside of English.

Built With

Share this project:

Updates