ASLTranslate

Home page
Translate page upon loading
"hello, my name is Sammy"
"I love hackathons!"
Quiz example
Quiz example answer

Inspiration

With the interests and passions to learn sign language, we unfortunately found that resources for American Sign Language (ASL) were fairly limited and restricted. So, we made a "Google Translate" for ASL.

Of course, anyone can communicate without ASLTranslate by simply pulling out a phone or a piece of paper and writing down the words instead, but the aim of ASLTranslate is to foster the learning process of ASL to go beyond communication and generate deeper interpersonal connections. ASLTranslate's mission is to make learning ASL an enriching and accessible experience for everyone, because we are dedicated to providing a bridge between spoken language and sign language, fostering inclusivity, understanding, and connection.

What it does

ASTranslation lets users type in words or sentences and to translate into ASL that is demonstrated by a video (gratefully from SigningSavvy). Additionally, there is a mini quiz page, where the user can test their vocabulary on commonly used signs. This provides the user with a tool to practice, familiarize, and increase exposure to sign language, important factors in learning any language.

How we built it

We leveraged the fast development times of Flask and BeautifulSoup4 on the backend to retrieve the essential data required for crafting the output video. With the dynamic capabilities of React and JavaScript on the front end, we designed and developed an interactive and responsive user interface. Leveraging FFMpeg (available as a library in Python), we seamlessly stitched together the extracted videos, resulting in a cohesive and engaging final output to stream back to the user. We utilized MongoDB in order to perform file storing in order to cache words, trading network transfers in exchange for CPU cycles.

Challenges we ran into

To ensure smooth teamwork, we needed consistent access to sample data for our individual project tasks. Connecting the frontend to the backend for testing had its challenges – data availability wasn't always the same on every machine. So, each developer had to set up both frontend and backend dependencies to access the required data for development and testing.Additionally, dealing with the backend's computational demands was tricky. We tried using a cloud server, but the resources we allocated were not enough. This led to different experiences in processing data on the backend for each team member.

Accomplishments that we're proud of

For many of us, this project was the first time coding in a team environment. Furthermore, many of us were also working with full-stack development with React for the first time, and we are very content with our successfully completed the project. We overcame obstacles collaboratively despite being our first time working on a team project and we can’t be more proud of our team members who all worked very hard!

What we learned

Throughout the building of the project, we learned a lot of meaningful lessons. Ranging from new tech-stacks, teamwork, the importance of sign language, and everything in between and combined. ASLTranslation has been a journey full of learning in hopes of learning journeys for others. All of our teammates had a wonderful time together and it has been an unforgettable experience that will become an important foundation for all of our future careers and life.

What's next for ASLTranslate

We will add more user-interactive features such as sign language practice, where we build a program to detect the hand-motion to test the accuracy of the sign language. To further expand, we will create an app-version of ASLTranslate and have it available to as many people as possible, through the website, mobile and more! Additionally, as English and ASL does not always directly translate, ensuring grammatical accuracy would be highly beneficial. Currently, we prioritize generating user-friendly outputs over achieving perfect grammatical precision. We have considered the potential of employing artificial intelligence for this task, a avenue we are open to exploring further.