Inspiration

Being a friend with a deaf person makes us appreciate how much communication is important and needs it in our daily bases tasks. As Software Engineers we believe that technology comes to solve problems and makes everything easier. The technology was never limited to young and smart people, with Technology no discrimination

What it does

Our innovative solution is to recognize sequences of hand gestures and facial expressions to translate either to sentences, words or alphabet that will be provided as text or speech as an output in realtime. Also, we come up with a hardware-less solution to reduce the cost of using our solution

How I built it

with Tensorflow's Machine learning we built a realtime Image classification model to recognize the sequences of hand gestures by incorporating the k-Nearest Neighbor (k-NN) classifier algorithm, also we used image processing techniques to preprocess the images and make it able to detect the hand gestures in varying Lighting and background Conditions to increase the confidence rate of our model. with embedded google vision SDK, we were able also to recognize sequences the facial expressions. At the end with the use of google voice API, our solution translates to a speech or text (TTS: Text to Speech)

Challenges I ran into

First challenge how to build our image classifications model and from where we will bring a set of images to train it (we spend all night taking a thousand images to train our model). Second, how to increase the confidence rate and make our model the most effective working with we had varying Lighting and background Conditions

Accomplishments that I'm proud of

Making our solution able to effectively track and recognize sequences of hand gestures and facial expressions with a high rate of effectiveness

What I learned

First in personal developments: we learn how to work effectively within a team, under pressure and until the last minute without giving up. Regarding technology: we learned using we used image processing techniques to preprocess the images, deploy apps using SAP Cloud, using google voice API.

What's next for WeTalk

Build a Video Call Plugins for Skype, Google Meet, FaceTime and Facebook to make it easier for deaf persons to chat and communicate with anyone in this world without limitation

Built With

Share this project:

Updates