Literacy Assistant Pro

Inspiration

Several members of our team have educators in our family, and one of the most difficult concepts to teach children and non-native English speakers is literacy. We decided that utilizing AI to assist speakers with literacy would allow learners to read and pronounce difficult words.

What it does

This project provides the user with a passage to read, and dynamically streams audio from the user's microphone, transcribing their words using AssemblyAI's realtime speech recognition tools. From this transcribed text, the incorrectly pronounced words are displayed to the user for review, providing them an option to listen to the correct pronunciation, and then try to replicate it.

How we built it

We build the program using Python, AssemblyAI's realtime speech recognition tools, and PySimpleGUI for a user interface.

Challenges we ran into

We faced significant struggles with installing the packages to begin using the necessary tools, working with the PySimpleGUI package to implement the user interface, and implementing socket programming to connect to the AssemblyAI endpoint, in order to transcribe the user's words on the fly.

Accomplishments that we're proud of

We are very proud to have developed a useful, functional, and finished project for this Hackathon. This was most of our team's first Hackathon, and our primary goal was to have a presentable end-product, which we feel we completely surpassed. Additionally, we wanted to utilize AssemblyAI's API for our project in order to begin exploring Machine Learning and Artificial Intelligence, which was a very rewarding part of the project.

What we learned

We learned how to use Git, integrate another API into our own program, utilize PySimpleGUI, and create customized algorithms from scratch. Further, we developed our skills and knowledge of the software development process, socket programming, and technical communication.

What's next for Literacy Assistant Pro

We plan to continue building this project into a more fully-developed program which can display entire passages at one time in the user interface, instead of only single sentences. Additionally, we plan to integrate a mobile app alongside the web project, which would allow the user to scan a picture of any passage with their phone camera, and learn to pronounce each of the words in their custom passage.