Check-in #3 update:
Introduction: Sign language is an essential tool used to communicate within deaf communities. While sign language is widely used in these communities, it is uncommon elsewhere and poses a problem for accessibility. This paper aims to develop a deep learning architecture that can accurately label ASL fingerspelling in real time to bridge the gap between normal-hearing and deaf communication. Almost all fingerspellings can be represented by static images, except for the “J” and “Z” signs. We can leverage this property to use convolutional neural networks to interpret and classify input fingerspelling images.
Challenges: Thus far, the most challenging part of the project has been preprocessing. Though the paper's authors describe the steps they took to preprocess the dataset images, in addition to various errors we have encountered, it was somewhat difficult to determine whether we are preprocessing "correctly" (or at least in a way that is beneficial to our model).
Insights: Our model is able to train, and our testing accuracy is ~99%. This exceeds our expectations! Going forward, our main goal will be to maximize the performance of our real-time classifier, which is still in progress.
Plan: We believe we are on track with our project. Our model's accuracy has already exceeded our stretch goal, and we have started implementing our real-time classifier. The bulk of our time will be devoted to the aforementioned real-time classifier. If needed, we may tweak our preprocessing, depending on how well the classifier does.
Log in or sign up for Devpost to join the conversation.