Since communication drives the world, it's essential that everyone is free to communicate their ideas with each other effectively. However, for the deaf and the mute, this is not the case. We seek to address this issue by giving everyone the ability to speak by translating ASL to speech. This way deaf and mute people can communicate with people that don't know ASL.

What it does

It translates ASL (American Sign Language) into English language by interpreting the user's hand gesture through electrical impulses to the hand along with data from an accelerometer and a gyroscope and translating it into English before outputting the audio through a speaker.

How we built it

We used Java to make an android app and had it connect to the Myo armband via Bluetooth. an app that would receive motion as input, translate that to text "behind the scenes" in the app, and output the text as speech.

Challenges we ran into

Receiving input: App would output user-inputted text as speech, but if user added more text while the app was still outputting the previous phrase, the app would immediately start outputting the new text as speech, disregarding the rest of the first phrase. Bluetooth: Under time pressure, our program couldn't connect the device to the Myo armband through Bluetooth. We solved this by giving the app permissions manualing.

Accomplishments that we're proud of

We pulled an all-nighter with minimal caffeine. Developed an app for Myo armband without prior experience Wrote code to connect to Bluetooth

What we learned

Through this project, we gained greater skill with Android Studio, developing for the Myo armband, and Bluetooth connectivity.

What's next for Myo ASL to Speech

-Incorporate machine learning to increase the repertoire and "vocabulary" -Increase accuracy of interpretations (detect more subtle movements) -Involve the community in order to further development (open-source) -Make it more user-friendly so the user does not have to constantly press start/stop -Expand to more languages -Allow users to macro their own phrases to their own gestures, giving them much more communication freedom.

Built With

Share this project: