-
-
Demonstrating sign recognition
-
Hello!
-
Our webapp can also be used to learn sign language
-
Correctly doing the "L" sign
-
Incorrectly doing the "L" sign by doing an "O" instead
-
Having others try to spell "Hello"
What is our Inspiration
We were inspired to work with people with disabilities, Sydney has a neice that is developmentally disabled and she has always wanted the opportunity to integrate her love for the community and her love for programming so we came together as a team to make that dream a reality.
What we built
Machine learning is a topic of great interest in the tech community for quite some time and is something our team an members have been looking forward to working with. We built a web based application that recognizes sign language from visual input that we taught clarfai and outputs the text and audio corresponding with each letter.
How we built it
We used the Clarfai API to train our models to recognize some letters. We used over 1,000 points of data collected throughout the hackathon from various people as shown in the video. We then submit an image of our hands to the server to process the image and return the letter that bests resembles our models. We then print the text based on a confidentiality percentage with Javascript.
Challenges we ran into
Effectively collecting data points for our various models and being able to parse through the results to retrieve the correct letter.
What we learned
Clarifai API, Neural networks, Artificial intelligence, and image recognition technology.
Log in or sign up for Devpost to join the conversation.