Our team wished to create a product that would not only incorporate innovative technology but also find a way to improve society. Through brainstorming, we realized that an overlooked issue in society is clear communication. This lead to discussing the community of signers and how it must be difficult to integrate into a society of fluent english speakers. Over 70 million people in the world use sign language as their first language. Our application, Sign Language Translator, allows users to communicate with others easier and helps to alleviate disadvantages in society.
What it does
This application translates American Sign Language (ASL) into text and speech in order to make communication more effective.
How we built it
The motion sensory data is gathered and interpreted using a Leap Motion and the first-party software, Orion. The code is written in Python 2.7. Google Text To Speech software was used to deliver text to speech, and VLC was used to play the .mp3 audio files. The words that are a part of the vocabulary are each individually processed to be recognised after very specific gestures.
Challenges we ran into
At the beginning, learning to interface with the Leap Motion hardware was tedious and time consuming, as no group members had any prior experience with it. However, the greatest challenges involved the implementations of the input actions that are involved in sign language. The SDK only provided us basic sensory tools, requiring us to write very detailed code to accurately perceive every gesture.
Accomplishments that we're proud of
The biggest accomplishment was being able to identify various signs and translate a user's actions. This was due to the difficulty of just detecting and tracking what movements the user was doing.
What we learned
Throughout this weekend, our team gained a strong foundation for reading information from a 3D interface. By applying our previous linear algebra knowledge, we were able to successfully break down specific actions and translate these gestures into understable code. In addition, we were able to further our python language experience while learning how to use Leap Motion's Orion software. Lastly, we also learned the importance of debugging and testing our code in order to improve efficiency and accuracy.
What's next for Sign Language Translator
Sign Language Translator is built for scalability. The code itself was written with reusability and modularity as top priorities. The next step for SLT is an expansion of its vocabulary base in order to better encompass the needs of American Sign Language (ASL) signers. In addition, a friendly user interface that is both simplistic and pragmatic will allow SLT to be accessible to the average user.