●Around 700,000 to 900,000 people across the world are deaf and dumb, which means they face difficulty in having normal conversation.
●The number of deaf and dumb in the world continuously increasing and they are introverted closed society. Therefore, Deaf-Dumb people do not have normal opportunities for learning.
●Uneducated Deaf-Dumb people face serious problem in communication with normal people in their society.
●To help them communicate with normal people and help them learn and get educated like normal people we thought of creating something which will help them communicate with each and every person without the need of a third person as a translator.
● Every deaf and dumb person want to communicate with other but they couldn't because of their inability and insecurity of being judged and being trolled for their disabilities. Because of this reason they are unable to make the best use of the skills that they acquire.
●Even this pandemic time of covid-19 is hard for them as classrooms have shifted from offline to online which makes it difficult for them to attend classes and study.
What it does
●It is a platform which is basically built for the deaf and mute people to help them in communicating and making their online classes interactive.
●The deaf and dumb people can use their sign language which in turn will be translated into a text message using the machine learning algorithms .
●The translated text will be conveyed to the normal person.
●Similarly as normal person are not aware about the sign language they can reply to the message using normal voice message or a text message which in turn will be translated back into a GIF or alphabet sign language pictures and will be shown to the deaf and dumb person .This will help them interact with each other.
●This feature will not only help the person to communicate but also the need of third person as a translator between the deaf - dumb person and the normal person is removed .
● Using this web app the classes can be made interaction for the deaf and dumb people will automatically increase their ability to understand the lectures hence they will be able to study like normal people without any difficulty in communication and will be able to enhance their skills and make the best use of the skills they have without being trolled for the reason of being deaf or dumb.
How I built it
Our Project is a Realtime Web Application that uses deep learning neural model which is deployed in the backend(breaked into two different module) for effective communication. ●First module consists of 574 gestures that are trained using deep convolutional neural network model build using keras and tensorflow with an accuracy of 90%.
●Then for Real time sign detection we use computer vision library called opencv and integrate that with both the module.
●In the second Module We implement Speech to text recognition using speech_recognition library and then convert into GIF accordingly.
●Tkinter for converting our Python code into web app
●Cloud service (Heroku) to enhance the scalability of the product for large scale user group
Challenges I ran into
There were many challenges which we faced while making this project. But one of them was to train the model to learn about various gestures used by deaf dumb people to communicate. Training the model to show exactly the same GIF or the set of alphabets when converting from text and Integrating the model with the web app so we can use it during video calls was also a difficult task.
Accomplishments that I'm proud of
We are proud that we were able to take a step forward to help the people who find it difficult to convey their feelings and represent themselves in the virtual world. This small step would add to their confidence to represent themselves.
What I learned
This project improved our ability of working in collaboration and teamwork. We learnt how to deploy our ML model in cloud platform .We also learnt how to create real time ML applications.
What's next for Sanchar
●Creating an integration in form of Plugins or external ‘Join as members’ for major video call platforms will enable the deaf-mute people to stand and represent them at every level.
●Multi-language speech recognition to communicate in different languages like Spanish, Hindi, French, German etc for the end users using our app or on the integrated platforms.