Indian Sign Language Translator

Inspiration

According to the world health organization, there are 466 million people have disabling hearing loss and 80% of deaf people are illiterate or semi-literate. Imagining people were not able to get proper education or communication because they were not able to hear or talk broke my heart. There isn't a single app that can satisfy the need for a sign language translator. It was shocking that there was no sign language translator app but there were a few apps which were similar to sign language translators like Hand Talk (which translates words to sign language ) but was not available in the google play store and in their official website it was available in trial versions.

What it does

It is an app that translates sign language to texts and it can even convert English text to any other language. It is an application of Deep Learning used for a social cause.

How we built it

A web built app using Streamlit that can translate sign language using TensorFlow and convert it into text. Users can also convert English text to their preferred language by selecting the language which is implemented with the help of Google Translator API. The dataset for each word is taken from the webcam. The sign for each word had been shown/tested, the gestures of the hands were captured using Mediapipe and the captured key points were stored in a NumPy array and saved in a file later the model is trained with the NumPy array, and here using the LSTM model from TensorFlow the procedure was moved forward. After training the model for each word, it was tested with the webcam.

Challenges we ran into

The data set was tedious to feed in. Conversion of the procedure from machine learning to an application was really difficult. Optimizing the Algorithm was a bit difficult. Accomplishments that we're proud of Prediction of algorithm Reliability of the app is about 96-98% Satisfaction of giving a solution for a global problem/ social cause.

What we learned

Acquired Deep Learning Knowledge. Learned Sign Language. Image/ Video Processing Processing Conversion of the procedure from a machine learning to an application What's next for CARE CLUB's - Sign Language Translator (Indian) More Languages to be added to the application. Text to sign language can also be developed. Increase the speed of the application.