SignAcademy

Inspiration

We wanted to develop a tool that would enable people to learn ASL without a need to physically interact with a teacher. While the basic ASL concepts are clear and simple, people might often display the characters in a slightly different manner -- which could be easily determined by our AI-powered assistance. SignAcademy enables its users to practice displaying the ASL characters and helps to further develop accessibility and world communication.

What it does

It uses a TensorFlow powered model that recognizes ASL characters only when those are correctly displayed by the user. In the future, we plan to add multiple features like gesture recognizing and personalized feedback so that our users will get customized advice on how to improve their language skills. Moreover, we also provide access to some educational resources and hope to foster an interest in learning sign languages.

How we built it

We pre-trained and built the TensorFlow model using Google's Teachable Machine project: we manually exposed the model to ~300 instances of each of the 26 character-based classes and tried to make the model work efficiently in terms of response time and quality.

Challenges we ran into

One of the hardest things was figuring out the variations of the characters in ASL and the ways to display them. While we had to practically teach ourselves the basics of the language, we tried to apply our technical skills to create an impactful tool that fosters learning and communication.

Accomplishments that we're proud of

We're proud of building something that is accessible for everyone and serves a greater purpose!

What we learned

We learned that communication is very important and that training models takes a lot of time and effort :)

What's next for SignAcademy

In the future, we plan to develop gesture recognition, improve our character recognition model, develop other internal educational resources, and partner up with other companies to provide them with access to our machine. We think that collaboration is the key to making things work on a global scale!

Built With

• css
• html5
• javascript
• netlify
• p5.js
• tensorflow