⚡ Inspiration

  • After watching a video about a deaf person being mocked and ridiculed at a drive thru, It made me open my eyes to the problems deaf people must face every day with communication. Therefore, I wanted to build something that could help them communicate more easily.
  • After many hours of brainstorming, I came up with the idea of a sign language translator that uses computer vision and machine learning to detect sign language gestures and translate them to english in real time

❓ What it does

  • It translates sign language gestures (only numbers for now) to english in real time using your computers webcam

💻 How we built it

  • I used open CV to build a dataset based on my hand gestures of over 5500 images on the numbers 1 to 10 in sign language
  • I then used tensor flow and keras to construct, train and test a convolutional neural network on the dataset of sign language number gestures
  • Then I used the live webcam and the trained model to detect and translate sign language numbers

💪 Challenges we ran into

  • I faced many challenges including small errors in my code that I spend hours looking for and changing as well as errors in the training of the model and losing lots of progress
  • I also faced challenges with the camera footage recording my webcam which wasn’t working, so I had to figure out how to use my phones camera as the webcam
  • I also faced challenges in the models training when the epochs would get skipped and the model only trained for 4/10 of the epochs

😀 Accomplishments that we're proud of

  • Coming into this project, as a beginner programmer, I knew almost nothing about Computer Vision and CNNs (Convolutional Neural Networks). I had to watch a video about it 5 times before I kind of understood a little about CNNs and had to work late into the night just to fix errors and get it done in time. However, I am glad I took part in this project because I learned a lot of useful skills I could use later on.
  • I was pleased of the final model which worked just as I expected (kind of)

🖐 What's next for Sign Language Detector

  • I aim on working more on, and improving the sign language detector to increase the accuracy and reduce the time taken to detect the gestures.
  • I also want to extend the functionality so that it can detect letters as well as numbers which I didn't have time to do in this hackathon

Built With

Share this project: