Speech Clarity

Inspiration

After reading through the United Nations Sustainability Development Goals we decided to make a project that would help people educate themselves. After coming up with the idea of making a pronunciation tutor, we realized that the text-to-speech voice had to sound more human and less like a robot to make a more engaging and enjoyable experience. After researching our options we landed on ElevenLabs, one of our sponsors, to make our tutor sound and feel more human.

What it does

Speech Clarity tracks and grades your pronunciation by listening with precision down to the phoneme. After you paste your text of choice and record your attempt, it compares your pronunciation to the most correct pronunciation of each phoneme and returns an overall score with went right and wrong.

How we built it

We built Speech Clarity using a repository on GitHub called OpenPronounce by Halleck45, Python, ElevenLabs API, and some frontend languages including CSS and TypeScript. By our efforts and these tools we were able to create Speech Clarity.

Challenges we ran into

Our biggest challenge was installing all the tools we needed. Trying to install the same tools on completely different environments was a mess and took up a good chunk of time for us. We eventually overcame this obstacle by creating set up scripts and using virtual environments to ensure that if someone had an issue we all had the same setup and could work together to resolve it. Another challenge was deciding on how forgiving our grading system should be. A combination of different microphones and audio drivers can significantly skew what is picked up versus what's actually said. Especially with a tool that has precision down to the phoneme we had to make our tool more robust by making our grading more forgiving but still strict enough to where it can pick up smaller mistakes.

Accomplishments that we're proud of

We are proud of being able to assemble an end-to-end project that works better than we expected. At first we were thinking of using large public APIs like ElevenLabs, although unfortunately many weren't speech-to-phoneme and free to use. We decided to commit and took the big leap to designing our own "API" which ended up working way better than we expected and was one of the bigger accomplishments we had with this project.

What we learned

It being our first hackathon, we picked up many soft skills related to groupwork such as communication and problem solving through collaboration. We also learned lots of new end-to-end development skills, organization and GitHub management, new python libraries, and how to work efficiently as a group.

What's next for Speech Clarity

Speech Clarity is soon going to see more robustness to things like accents and also a significant increase to its range of languages. With many more tools out there, we are able to bring them together and continue expanding Speech Clarity to a larger variety of languages and accents.