Inspiration
Videos have been coming up on youtube where deaf people have been having trouble ordering at drive thrus because of the pandemic. We wanted to address this issue, so the employees can communicate easier.
What it does
Transcribes sign language letters into words.
How we built it
Flask for backend, and P5.js + HTML for front end. CNN to recognize sign language letters using the MNIST database.
Challenges we ran into
We wanted to do localization. Datasets and bugs killed that idea. We wanted to animate text to sign language, interfacing with flask made that difficult, and there was no database, animations would have taken too long.
Accomplishments that we're proud of
Flask + P5.js. Our CNN getting a high accuracy on the training data.
What we learned
Get and Post requests for Flask. Datasets take a long time to make. Debugging with limited sleep is hard.
What's next for Signome
We want to make it into an app with more sign language vocabulary, and finish the animations.
Log in or sign up for Devpost to join the conversation.