Inspiration

According to the World Health Organization (WHO) five per cent of the world's population live with disabling hearing loss: that's more than 360 million people. According to the General Authority of Statistics in Saudi Arabia, KSA has the highest recorded number of deaf people in the Arab world with more than 2 million people. These statistics inspired me from my bottom of my heart to work on an integrated and efficient AI project to bridge the communication gap between us as normal people and the deaf people.

What it does

The system translates Arabic sign language to Arabic speech and vice versa. Recorded presentation: https://youtu.be/wDKdpaAPf80

How we built it

By using YOLO-v5 to detect and recognize Arabic sign language. On the other side, I use Klaam which is a python package to convert speech to text. Then, I use regular expression to map the text to the equivalent sign which is prerecorded.

Challenges we ran into

  1. Resources of datasets.

Accomplishments that we're proud of

  1. I'm proud of creating PoC dataset from scratch.
  2. implemented a PoC system that translates Arabic greetings (Hello, good morning, etc.).

What we learned

  1. The concept has been proved.
  2. We need more signers participating in building the dataset for better training.

What's next for Arabic Hands Talk

  1. I assign at least two signers to for bigger datasets.
  2. Deploy the system as a mobile-based app as a beta version.

Built With

Share this project:

Updates

Submission history