Dumb people cannot speak an oral language or have some degree of speaking ability but prefer not to talk because atypical voices often draw negative or unwanted attention. Using sign language, these individuals communicate. But not everyone knows ASL who will understand them. Due to COVID-19, if a Dumb person somehow tried to speak, they can't because they're wearing masks. So, what's the resolution?
What it does
Speakable, where we tried to create a user-friendly web application in which a Dumb person can record their sign language hand gestures, name them and it will convert them to Sound.
How we built it
- TensorFlow and TensorFlow.JS
- Open CV
Challenges we ran into
To implement model training on client-side and choosing which Deep Learning algorithm to be used. Also, how to convert live text to speech.
Accomplishments that we're proud of
We're able to train the model on the client-side with a good accuracy without losing frame-rate.
What we learned
What's next for Speakable
To improve the UI/UX by shifting the code base to React.JS. Also, create a Profile Section for each user and able to store their trained gestures and not training again on Cloud. Lastly, add more voice options with live translation!