Inspiration
We were inspired by a close friend of ours within the ASL community who has difficulty communicating with others who don't use ASL. The goal of our project was to help bridge the gap between ASL and non-ASL speakers, facilitating more inclusive communities.
What it does
Sign Me Up! is an application that helps translate common phrases and letters from ASL to English text allowing people to both learn and understand ASL better.
How we built it
We built it by creating an image detection software that recognizes over 40 different signs from words like "Hello," "Eat," and letters A - Z. We gathered data by taking images of us doing all of the signs and trained it using RandomForest.
Challenges we ran into
We initially had a lot of difficulty learning sign language, as we had limited experience outside of basic phrases. As we developed our detection models, we had difficulty with some signs that were being recognized incorrectly, which we were able to fix with better model parameters and rerecording some of the signing image data.
Accomplishments that we're proud of
The biggest accomplishment we were proud of was when we successfully saw our fingers getting outlined as well as having the ability to translate full sentences! It was very rewarding to see our models improve as we added more detectable vocabulary and phrases to the training data.
What we learned
During the development of this software, the team gained experience in recognizing and signing specific ASL letters and phrases. The project also provided valuable insight into training machine learning models to accurately interpret and differentiate between various signs.
What's next for Sign Me Up!
We hope in the future to add more words and phrases to our collection as well as the ability for it to use text to speech to translate the words spoken in ASL.
Log in or sign up for Devpost to join the conversation.