Inspiration
The popularity of language education apps such as duolingo make learning new languages a fun an interactive experience. However when comes to ASL , apps are typically purely in matching format rather than the user interacting to practice the language.
What it does
Gesture takes the ASL learning experience and allows the user to actually perform the signs to receive realtime feedback.
How we built it
The web portion of the application utilizes Next.js framework for frontend , firebase for user data and authentication , and flask for running the mediapipe gesture recognition model.
Challenges we ran into
Training the ASL model was incredibly troublesome as it frequently crashed and we were limited by the performance of laptop.
Accomplishments that we're proud of
For most of our team , this was our first time working with an ml model and getting to work on a web application.
What's next for Gesture
Gesture has future plans for adding more modes such as an endless mode that allow you to continue signing until failure.
Log in or sign up for Devpost to join the conversation.