Inspiration

Most deaf children are born to hearing parents. This app helps parents learn ASL to communicate with their child, while also building the foundation for their child to connect with others.

What it does

Our app presents one of 563 words and uses a media-pipeline ML Sign Language Recognizer to check if the user signed it correctly, helping parents practice ASL and reinforce learning.

How we built it

We built the frontend using Vite + React and the backend with Firebase, hosting the app on GitHub Pages. The app leverages the Sign Language Recognizer (SLR) from PopSign Labs, powered by the GTK-Web toolkit created by Nana Gupta.

Challenges we ran into

We faced challenges integrating the existing Git build into our frontend and handling merge conflicts between teammates. This taught us the importance of frequent commits, pulls, and clear collaboration.

Accomplishments that we're proud of

We successfully integrated the Sign Language Recognizer (SLR) into a web-based application with a fully functional Firebase backend, creating a seamless, interactive experience for users.

What we learned

We gained hands-on experience with React, Vite, and Firebase, learned to integrate existing ML tools into a web app, and saw the importance of team collaboration, version control, and iterative development.

What's next for Significance

We plan to add score tracking and global highscores for all users, along with a system to save and categorize signs as “aced,” “needs practice,” or “needs studying” to personalize learning.

Share this project:

Updates