Motive
My motive is to empower individuals with hearing loss by providing them with a tool to communicate effortlessly with the world around them. Communication is the cornerstone of human connection, and denying individuals the ability to express themselves not only isolates them but also deprives society of their unique perspectives and talents.
ASL (American Sign Language) is not merely a form of communication, it is a language that embodies the rich culture and history of the deaf community. Learning ASL is not only an act of accessibility but also an acknowledgment of the importance of embracing diversity. It opens doors to education, employment, and social integration for individuals with hearing loss, enabling them to thrive and contribute meaningfully to society.
Through the ASL app concept, I hope to increase awareness about the importance of ASL and promote its learning among a wider audience. By providing a user-friendly platform that offers translation services and educational resources, the aim is essentially to bridge the gap between the deaf and hearing communities, fostering understanding and inclusivity.
Together, let us embark on this journey towards inclusion, breaking down barriers, and celebrating the beauty of diversity. By empowering individuals with hearing loss through the universal language of ASL, we can create a world where everyone's voice is heard and valued, leaving no one behind.
What it does
ASL Buddy has multiple features, notably, it's Chat Assist, which provides a speechtosign and signspeech converter/translator so people can communicate and bridge the gap between not knowing and knowing ASL. ASL Buddy also has a Study feature which provides quizzes, so an individual can learn ASL!
How I built it
I created the app mock-up on Figma and created specifically both speechtosign and signspeech converter/translators.
For the speechtosign I used Javascript, HTML, CSS, and Python, to create local application that picks up audio recording and gives back sign language in correspondence with each word spoken and highlight each letter.
For the signtospeech I used OpenCV, Mediapipe, and TensorFlow, to create an application that picks up live capture through a camera, in my case my webcam, and recognizes sign language at 97% accuracy and provides its corresponding letter. The purpose of this application is so that someone who doesn't understand ASL can use it to understand someone.
Challenges I faced
I never really trained a dataset before which I had to do for the signtospeech component, was challenging and quite long but I overcame it.
I don't have much experience in JS, HTML and CSS, so was challenging.
The accuracy of my speechtosign was incredibly low, but I go fixing it.
Accomplishments
Very proud of the accuracy on both speechtosign and signspeech converter/translator, and how they look.
What's next for ASL Buddy?
Hopefully creating the study component and actually develop the app. Possibilities are exciting and I'm very much looking forward to continuing to work on this project.
Log in or sign up for Devpost to join the conversation.