ASL Live Translation

Inspiration

The inspiration behind developing an ASL code interpreter is to enhance the accessibility for the Deaf communities. An ASL code interpreter is a tool that empowers Deaf individuals to communicate more seamlessly in various aspects of their lives, including education, work, and social interactions.

What it does

In the process of creating our project, we were able to gain a good understanding of comprehensive ASL vocabulary learn the various signs, gestures, and expressions used in ASL, and understand the meaning and context behind different signs as we build the program. We were also able to grasp a good understanding of machine learning basics, as they play a crucial role in training models to recognize and interpret signs. Understand how to build, train, and optimize models for gesture recognition.

How we built it

The project starts by setting up a special computer setup using Python, a popular programming language, because it has a lot of helpful tools for this kind of work. First, we use a tool called OpenCV to get video from the computer's camera. OpenCV helps us change the video so another tool, MediaPipe, can understand it. MediaPipe is really good at seeing where hands and fingers are in the video. Then, we use TensorFlow, another smart tool, to figure out what the hand movements mean in sign language. After we know what sign was made, we change it into spoken words using either gTTS, which needs the internet to work, or Pyttsx3, which doesn't. Lastly, we use Pygame, which is usually for making games, to play the spoken words out loud. This whole setup helps turn hand signs into speech that we can hear.

Challenges we ran into

One of our biggest challenges would have to be training a machine learning model for ASL interpretation that requires a large data set of signers. The accuracy of the hand recognition system was also something we faced difficulties with.. Achieving high accuracy in identifying specific signs and gestures was difficult as we had issued with our webcam.

Accomplishments that we're proud of

The ASL interpreter empowers Deaf individuals by providing them with a tool that furthers their ability to communicate effectively in various aspects of life, including education, work, and social interactions. We are proud of how far we advanced in our code .