Inspiration
When we entered this hackathon our goal was to create an ASL translating tool which could be used to translate ASL phrases into English phrases. We got our inspiration from one of our friends' co-worker, Ali who is an ASL speaker and works online. So we could only imagine the daily struggles Ali has to face communicating to his co-workers. Thus, we decided to create a tool which people like Ali could use to experience a smooth line of communication.
What it does
Our project caters to those who are unable to speak and thus, use ASL to communicate with their peers. Since due to the global pandemic all in-person activities are now online, we decided to create a aslX. aslX takes multiple ASL inputs and displays their respective English words/phrases on to the screen. This way individuals like Ali can communicate with their co-workers without having trouble getting their point across.
How we built it
We used MediaPipe Hands API and we are using Finger Pose from Andreas Schallwig to create our ASL translator.
Challenges we ran into
Trying to track dynamic motion. Trying to do this at a high frame rate. Trying to compile a large library of hand signs to translate in a short amount of time. Some ASL signs were interfering with ones we had implemented.
What we learned
MediaPipe is a very useful tool when it comes to tracking objects and their motion. Creating a framework can be helpful when tracking various things.
What's next for aslX
Smoothless dynamic motion detection.
Built With
- mediapipe
- vscode
Log in or sign up for Devpost to join the conversation.