Sign Language Recognizer

Inspiration

In our countries, sign language is a mere concept with absolutely no effort to promote its usage or importance. It was a big culture shock for us when we found out that news channels, for example, provide sign language translation for speeches. For this reason, we thought that as aspiring software engineers, we can play the first part in promoting sign language. Using this translator of sorts can help people understand sign language better and can make communication easier amongst people. What we truly set out to do was to add voice output for the input provided, but we were unsuccessful in extending our project to that level.

What it does

As mentioned above, the program observes a video and the hand gestures in that video (this can very easily be changed to accept real-time gesture inputs) and tries to interpret them, displaying the associated alphabet.

How we built it

We utilized PyTorch and OpenCV to build our project the program was divided into different classes for processing, training, evaluating, and finally running.

Challenges we ran into

The challenge truly was to get the recognizer to differentiate between similar hand gestures. We accept that our final result can not properly identify some of the alphabets.

Accomplishments that we're proud of

We were happy to observe that our hard work was paying off as for most of the alphabets, our recognizer was able to identify the correct alphabet. It is worth mentioning here that this was our first hands-on experience with building a machine learning project, for which reason it was a moment of joy when we saw some fruition in our result.

What we learned

We learned quite a lot about the power and applicability of machine learning and the many uses of libraries in Python!

What's next for Sign Language Recognizer

As mentioned, the next step is to first fix the program to identify all the alphabets correctly, including J and Z (which cannot be currently identified as they are moving gestures). Additionally, we would like to identify words and output the result in voice form.