Inspiration
Thought about real world problems which affected a substantial population of people. An extremely small amount of people (other than deaf/mute users) use sign language. Placing oneself in the place of someone who can only communicate via sign language is particularly harrowing. Imagine no one else speaking the only language that you know in a place you call home!
What it does
It takes sign language input from the webcam and converts it into English by matching and comparing the input image with a given dataset.
How we built it
Made using only python using the openCV library. Dataset stored within main computer and input matched to dataset images using ORB and brute force matcher.
Challenges we ran into
The input image was being matched with the dataset incorrectly due to changes in lighting and background. Solved by increasing the dataset by four times and including photos in the dataset in different lighting and in front of varying backgrounds. The computer takes the average of similarity with every single image and compares it to a threshold to output the text.
Accomplishments that we're proud of
Was able to fix most of the issues and errors that occurred without taking help from the internet! Learnt a lot about different python libraries and the concept of CV.
What we learned
Learned about the ORB and the brute force matcher which initialize and image with key-points and descriptors which later are compared to given dataset images to determine similarity. First and foremost, learnt about problem solving and of course, learnt the basics of sign language!!
What's next for Sign language to text
Project can be upscaled by converting it into an app for easy and quick translation. Webscraping can be integrated to even further the size of the dataset and increase accuracy by a huge number.
Log in or sign up for Devpost to join the conversation.