SignSpeak

Inspiration

Over 3.2 million Canadians have hearing impairment which can often create communication barriers and limit opportunities for people who are deaf or hard of hearing. As a result, this creates a stigma related to hearing loss and causes people to deny or hide progressive hearing loss.

Sources: https://www.cmaj.ca/content/192/50/e1809 https://www.ncbi.nlm.nih.gov/books/NBK207836/#:~:text=Others%20have%20difficulty%20accepting%20their,related%20to%20their%20hearing%20loss.

What it does

SoundSpeak is a web-based English/ASL translator that allows users to translate English to ASL and ASL to English. Given a typed message, the website will return images of ASL corresponding to each English word. To translate ASL to English, we implemented a front-facing camera screen that allows the user to record themselves signing a message in ASL which is translated to English (this functionality is in progress).

How we built it

Initially designed with Figma, implemented with HTML, CSS, Javascript

Challenges we ran into

We ran into challenges learning to use GitHub efficiently. We ran into problems related to overwriting code, creating duplicate code, pulling bugs, and losing code after pulling.

Furthermore, we realized that ASL is much more intricate than it appears. It has different grammar rules from English and cannot simply be translated using just the ASL alphabet. We also learned that ASL also relies heavily on facial and body language.

Accomplishments that we're proud of

We are proud of producing an app with functionality and a cohesive colour scheme.

What we learned

We had to learn how to work with three different languages at the same time and that it's okay to not know things. We learned how to use online resources more effectively.

What's next for SignSpeak

In the future, we would like to add a database of English words that have existing ASL signs to make our translations more accurate. We would also like to implement a machine learning model that can learn to read ASL signs from a submitted video from the user.