Sign Language Translator

• 

  This identified a W!

• 

  This is a sign language U

• 

  We are able to spell out a word and then get a TTS to say it out

Inspiration

We began our journey by first searching for various problems that lay with those in disability groups. One of these problems is the difficulty of learning ASL for those without the need for it. Throughout my life, our group only has had the chance to meet someone with a hearing disability once. During this encounter, despite having taken short lessons on the ASL alphabet when we were younger, we were at a literal loss for words when it came to communicating. So, this project was developed in hopes of deaf/mute individuals having an easier time talking to those who don't speak the language without having to type out words - something they may not be used to doing.

What it does

Our product uses webcam access to allow an individual to sign letters to either add to a word or directly be displayed to another user. We've added accessible and easy-to-use key binds to allow a user to add letters to a word, remove a letter, add spaces, etc.

How we built it

Using exclusively Python, and a machine learning model called TensorFlow, we trained our model with 87,000 images of the ASL alphabet to help get the most accurate interpretation possible. To also add text-to-speech to our program, we used the pyttsx3 library in Python to directly speak out the words.

Challenges we ran into

The biggest challenge we faced during this hackathon was the lack of time and resources. Training a machine learning model with so many images is a process that is not only time-consuming but also consumes many CPU resources. Our group believes that given more time, we would've been able to further train our model to make it even more accurate, but also develop the ability to interpret entire words - not just letters.

Accomplishments that we're proud of

However, given this time restraint, our group is very proud of our ability to complete a functioning model that is able to interpret all 26 letters of the alphabet rather accurately.

What we learned

We were able to develop our proficiency with machine learning, and found the experience very valuable to us for any future products that we may develop using similar processes.

What's next for Sign Language Translator

We have big goals for this ASL translator! We're hoping to advance it from simply interpreting letters to being able to understand and translate entire words, but most importantly, we plan to spend lots of time giving the model data in order to make it even more accurate than it already is.

Built With

• kaggle
• python
• tensorflow