Every gesture speaks.
My little brother suffers from anacusis, or in other words, hearing loss. Not just the little guy, but around 70 million people over the world are hearing impaired, and in just the United States, around a million people communicate using the American Sign Language. Surgical procedures happen to be really expensive, and day-to-day life gets a little complicated. Moreover, tech for social good is something we collectively believe in.
What it does
The user puts on the Myo armband and makes gestures. The android app SignSpeaks scans the Myo, syncs it, and records these gestures internally. Using the Myo Raw Data Capture, we log the respective EMG, accelerometer, and gyroscope values in a .csv file which corresponds to a specific text character or string in the English & French languages. For the purpose of demonstration, we've used the implementation for every programmer's signature first words - "Hello world." Using the Nuance and Google's text-to-speech APIs, this text gets converted to speech, thus serving as an effective interpreter, making communication between people breezy.
How we built it
We built a native Android app, and integrated the Myo, Nuance, and Google text-to-speech APIs.
Challenges we ran into
Getting the Myo set up and running was the biggest problem of all. Initially, there was a lot of noise and interference while recording the gestures. Also, integrating multiple APIs into our app was a bit confusing initially, but we figured it out eventually thanks to the help of team collaboration and asking for mentor help when needed.
Accomplishments that we're proud of
We made use of the raw data available and after a lot of tinkering, figured out how to record the gestures effectively. We made it bilingual - English and Canadian French!
What we learned
This was our very first time with hardware - especially the Myo armband, and we've learnt quite a bit about sensors and accelerometer graphs.
What's next for SignSpeaks
We're looking to fine-tune our hack, and also do another version with the LeapMotion. There's so many more functionalities that we can add to this, and that's what we'll be working on.