Hand Gestures to Speech and Text Conversion

Inspiration: deaf and dumb people cannot hear and speak. They face lot many challenges in their day to day life. In order to bring a change we thought of doing something which makes a change in their life and revolutionizes the technology.

What it does: It captures the hand gesture given by the user(input), extracts the required part of the image and then converts it into text and speech which is readable as well as audible.

How we built it: We have used the concept of CNN. The features of the images of hand gesture are extracted using two methods, namely, hand contour and complex moments. Classification using

Artificial Neural Network (ANN), where the recognition rate is calculated for both hand contour-based ANN and complex moments-based ANN and comparison is carried out. These numbers are optimized for EV detection, based on the ground truth we had, in the supervisory training dataset. Here, we used minimum error decoding to optimize filter values. Once the filter is tuned by supervisory learning.

Challenges we ran into: We faced a little challenging when we are making out training and testing dataset and converting them into grey scale images.

Accomplishments that we're proud of: We are proud that we made a successful step in brightening the life of many people through this experiment.

What we learned: We learned implementing the Artificial Intelligence and Machine Learning techniques as the live applications.

What's next for Hand Gestures to Speech and Text Conversion: It can be enhanced through future enhancements by adding additional features to it.

Built With

• ann
• cnn
• cnn-framework
• edge-detection
• ev-detection
• numpy
• pandas
• python
• supervised-learning