ASL Translate

Main page

Inspiration

5% of the world is deaf, but only 1% of the world knows sign language. This is an IMMENSE global problem because, without sign language, the barrier between the 5% of the world population that is deaf and the 95% of the world population that is not deaf is insurmountable. Converting sign language to text/speech has been done before, and while this allows deaf people to communicate with non-deaf people, it does not solve the entire problem. We wanted to make a website that can not only allow deaf people to communicate with non-deaf people, but non-deaf people to communicate with deaf people, removing the barrier entirely.

What it does

This website has 2 functions: First, you can sign words and sentences into your camera and it will be spoken by the computer live. Second, you can speak into the computer, and it will be converted to text live as well. This is 2-way translation between ASL and English.

How we built it

We built it using the standard web languages of HTML, CSS, and JS, but along with 2 main extra components: Web Speech API for speech recognition and text to speech, and TensorFlowJS for hand pose recognition and AI.

Challenges we ran into

We were originally planning on doing web scraping with NodeJS but ran into incredible difficulties. We spent 12+ hours trying to work with NodeJS but in the end ran out of time and had to just convert speech to text, instead of our original plan of speech to actual videos of sign language.

Accomplishments that we're proud of

We are proud of using TensorFlowJS to make a working sign language/hand pose detector, and we are proud of ending up with a finished project despite difficulties.