CoSign

Inspiration

We wanted to grow our nonexistent knowledge of deep learning while adding accessibility and community to people's lives. Our ultimate goal was to be able to have one person sign in one room and have what they said be spoken to someone else in a different room.

What it does

Our program identifies American Sign Language and prints it as text on the screen. We embedded the Google Text-to-Speech into the original algorithm so that the text will automatically convert to speech.

How we built it

This project is a sign language alphabet recognizer using Python, openCV and tensorflow for training InceptionV3 model, a convolutional neural network model for classification. The model itself was created by @loicmarie on Github, and we used and expanded his model to train on our device and connect via google cloud to the Google Text-to-Speech API.

Challenges we ran into

Processing power on laptop was slow, so algorithm training took 7 hours. Accuracy of classify algorithm is low because the data set we included did not have enough size and color diversity. Attempted to connect text to speech Google API to the Google Home mini, but encountered difficulties setting up the Google Home Mini connectivity. (also, literally everything)

Accomplishments that we're proud of

We are proud that we trained a machine learning algorithm with a massive data set! We are also proud of our resilience when encountering numerous roadblocks, as this is our first machine learning project. Finally, we are most proud to work on something that tackles an issue surrounding inclusion!

What we learned

We learned so much! We learned what and API is, and how to integrate it into a project. We also learned the process of training a machine learning model with a large dataset and we also learned a lot about terminals and troubleshooting.