ASLtoText

Inspiration

We wanted to make an application that was not only innovate, but one that could help others learn more about ASL while also acting as communication tool that aids discussion between people who otherwise might have a hard time communicating.

What it does

ASLtoText not only translates sign language to text real time using AI models, but also allows for users who don't know ASL to speak into a microphone, translating their words into text for people to read.

How we built it

We built our ASL to text translator on the Mediapipe gesture framework, and used openAI's whisper for audio to text translation. We used python, as its one of the quickest languages to set up AI models

Challenges we ran into

Due to challanges with Mediapipe Model Maker and outdated an outdated tensorflow, we were not able to make the AI gesture model as efficient as we would have liked. Even so, we were able to source code online for our gestures, and got our translator working.

Accomplishments that we're proud of

Managing to create both a real-time Audio and gesture translator using AI frameworks we have not previously worked on.

What we learned

Importance of teamwork :) The complications that can come up when working with AI modelling frameworks. How to work around problems and come up with new solutions to overcome the shortcomings.