SignAI

Inspiration

According to WHO, 466 million people worldwide suffer from some type of hearing problem which is 6.1 % of the world's population. In Nepal, around 16 % of disability-related cases are of deaf/hearing problems.

Most of the deaf people do not attend schools. There's very little awareness related to Sign languages. We are all aware that the credibility of their intellectual capacity is simply ruled out. We realize that it's mainly because of difficulties in communication amongst people. It's high time that we realize their capabilities and this is possible when communication process gets streamlined .Hence, here's where our project 'SignAI' comes to play.

What it does

1. Real-time Sign Language Detection: With a self-trained model, our app will use the users camera to capture hand gestures and recognise English words/phrases. We aim to feed the most common english words and phrases and gradually improve the model.

2. Interactive games Learning can be fun too. We'll utilize this model to build some fun quizzes, games like word of the day, guess the word challenge etc. Progress of the users will be recorded too.

Challenges we ran into

In the course of development, we ran into a lot of problems. We pivoted a lot due to this errors. From finding the right tool to collect and process data, to using the right model. The main challenge was to deploy the model to a right format and use it in our web platform.

Accomplishments that we're proud of

Apart from all the challenges that we dealt, we are very proud of the outcome we got. Our project has model with 75%mAp and we built it with our custom data. We have interactive and beautiful UI as well as quizes and other games to make the learning process fun. But most importantly we are proud of working as a team to create this in this short and competitive time.

What's next for SignAI

We tend to make model more accurate for realtime prediction and increase more feature for our learning platform

Built With

• auth0
• express.js
• fastpi
• javascript
• python
• react
• roboflow
• yolo