Inspiration

Total Population of India is 1.4 Billion People and out of which 7 -10 million people are hearing and speech impaired. The shocking part is there are only 250-500 certified translators for them. These people are left far behind in today's world of video meetings and work from home environment. So, we want to help them in our own way. Translation is repetitive task and can be easily solved with technology Source: pri.org

What it does

It converts American sign language to text in order to make it an easy process for them to communicate with everyone during video meetings.

How we built it

We trained a deep neural network on MNIST Sign Language dataset using Tensorflow. Then we converted this model into an API and rendered it through Flask backend and WebSockets. Finally, we made a UI for an end to end proof of concept.

Challenges we ran into

  1. Accessing client-side camera using Flask
  2. Converting TensorFlow model to an API and we can serve to our Flask backend
  3. Building modular UI. (Our team lacked that skill :P)

Accomplishments that we are proud of

Finally got everything working without breaking the code. :P

What's next for ASL Translator

  1. Audio conversion of text generated
  2. Chrome/Firefox plugin for overlay on apps like zoom and meets
  3. Make it mobile native for translation on the go

Built With

Share this project:

Updates