ASL Translator

Inspiration

Our friend with auditory impairments who struggles with communication in day to day society and often feels isolated and struggles to hear his family. This problem affects 1.5 billion people world wide and there is often very little accessibility and accommodations incorporated for these people. When you think about a fire alarm or emergency alerts, you think about loud noises and panic, these emergency signals rely very heavily on auditory cues, videos in airports, train stations, and other public places don't often have proper captioning.

What it does

Our project uses neural networks to identify ASL hand signs made by the user and seamlessly translates them in real time to English for easy comprehension by a person who doesn't know ASL

How we built it

We found a Convolutional Neural Network(CNN) to trace a person's hand position, we implemented a feet-forward neural network on top of that to interpret the hand positions identified by our CNN and translate that into everyday English.

Challenges we ran into

We had issues installing the required libraries, wifi issues, and due to the time constraints we weren't able to give the model adequate training for it to achieve the quality and accuracy we want the deaf community to have access to.

Accomplishments that we're proud of

We are incredibly proud of the volume of work we've gotten through with our efficiency given the time constraints, but most of all we're incredibly proud of the end result, which is a program that we made that can help the deaf community feel included.

What we learned

We learned a lot about neural networks, libraries that help make neural networks such as tensorflow, matplotlib, opencv, etc.

What's next for ASL Translator

We would love to train the model more to make it better and improve it's accuracy so potential users can have the best results possible. Also with this being a categorical neural network, we only have select options of what a sign could be, so we would love to add more ASL signs to recognize.