Considering communication is the most important key to perform almost every task, we decided to give everyone a way to talk even if you do not understand sign language.

What it does

It converts the sign showed to the webcam/leap motion and gives an output which is already learned by the machine. This helps to understand some specific letters/gestures.

How we built it

Building was a hard part since we were not at all used to the coding language of Wolfram Alpha. Connecting leap motion to Mathematica was tough, but we were able to figure it out and add gestures and add data samples to Mathematica. Then, we used the "classify" command and brought every letter in a single directory for the machine to learn.

Challenges we ran into

Setting up leap motion with Wolfram One was challenging. We ran into a lot of problems and also faced multiple dead ends there. Later, we faced problems with adding data/gestures to the machine and leap motion since it took a lot of multiple tries and hardware complexities.

Accomplishments that we're proud of

We are really proud of the fact that we can convert sign language to text output on a computer screen in real time and use less yet high-end technology.

What we learned

We learned how robust and capable Wolfram One Mathematica is since we did not know about it. Learning that coding and working of leap motion added to the learning experience.

What's next for Sign Language to Text using Machine Learning

Combining both leap motion gestures with real-time machine learning and Wolfram would enable us to collect better data and give a better outcome. Also adding speech to the outcome would enable blind people to understand.

Built With

Share this project: