We were actually working on an entirely different project and attempting to figure out how to properly parse the incoming data from the Kinect when we came up with this idea. While waving our arms wildly and monitoring the Kinect output, one member of the group joked that we should do sign language and see what happens. Turns out it was actually a great idea. In addition, some group members were inspired to develop this by the fact that 5% of the world's population suffers debilitating hearing loss, which includes 1/3 of the population over 65 years of age.

What it does

Essentially, Sign Me improves ease of communication between individuals who do and do not know American Sign Language. The idea is that one person will wear a camera which will record another person signing. Using an open-source interpretation algorithm, the signs are translated to English and then sent to our script hosted by IBM Bluemix, when another script also runs, sending the message to the recipient's phone.

Challenges we ran into

The installation of the devices, along with multiple software/language installation issues, especially with Python and Pip. In addition, we went through multiple languages in an attempt to get a way to work not only with getting video input, but also reading the English words that the sign language translated to. Also, our original way of handling messages ended up having to be replaced within the last hour.

Built With

Share this project: