We wanted to make it easy for the 430 million deaf and hearing disabled people across the world to be independent of translators for their daily communication needs. We created Signchat to work as a personal translator to ease communication and erase sign language barriers.

What it does

Signchat uses Teachable Machine to create the machine learning code. It recognizes hand signals and converts the signals to English. Right now we only have introductory phrases but plan to make it a fully functional sign-language conversation app.

How we built it

For the front end we used HTML, CSS, JavaScript, and Bootstrap to create our website. Our backend database was created using MongoDB. We worked together to figure out what aesthetic and pictures we wanted to have our website showcase, and implemented everything with our front end tools. For the back end we build the machine learning code using javascript, and taught it to recognize various phrases.

Challenges we ran into

We tried to just use python, django, postgres, html/css and upload that code using heroku in our initial attempt. After spending a lot of time trying to configure our github to a heroku pipeline, we decided it would be easier to use MongoDB for our backend. Our machine learning code was really fun to work with, and with more time we could have taught it more phrases.

Accomplishments that we're proud of

This was the first time we were able to combine the front end and back end successfully. We are also really proud of our machine-learning code, that was able to detect users hand movements and map them to sign language.

What we learned

We learned a lot about machine learning, pipelines, ci/cd, and databases.

What's next for Sign Chat

We would like to expand our website to include a mobile application, and to have additional features, like having a text-to speech function as well. We would like to fully deploy the site also.

Share this project: