Inspiration
What it does
SignSpeak translates ASL letters into English letters and leverages the Gemini API to eventually construct words. SignSpeak is particularly useful in applications like ASL proficiency assessments, assisting a first-time learner in growing their knowledge, and situations where text-to-speech communication is insufficient. While SignSpeak primarily does letter-by-letter translation, we aim to grow SignSpeak into a fully fledged ASL to English translator.
How we built it
We used Mediapipe and Tensorflow to design a fully self-trained ASL predictor model. We pulled an online dataset of images to train on. The frontend stack is built in Next.js.
Challenges we ran into
We found significant challenges in building model accuracy. We spent a significant amount of time tuning the model, and while it did produce results, it limited our ability to expand SignSpeak more.
Accomplishments that we're proud of
Strong confidence rates in letter detection by the model.
Log in or sign up for Devpost to join the conversation.