HandReader

HandReader poster

Inspiration

One of our team member's cousin, who is deaf and mute, can only communicate through sign language. Due to his condition, we thought of a way to make it easier for him to express himself to others.

What it does

The program recognises the hands of the person in front of the camera, traducing each sign to a letter of the manual alphabet.

How we built it

We built it using OpenCV which is a popular open-source software library used for computer vision and image processing; and MediaPipe, an open-source framework by Google that makes it easy to build real-time, AI-powered perception pipelines—especially for things like hands, face, body, and gestures.

Challenges we ran into

Learning how to apply hand recognition to cameras was quite difficult, as we did not know how to code on Python and configure the position of each vertex that corresponds to each finger and phalange.

Accomplishments that we're proud of

We have tested the project with other deaf persons and they were pretty satisfied with the final result, claiming that it made communication with people that did not know their language much easier.

What we learned

Being born as a deaf/mute person is much more difficult than it may seem. In our current society, communication is one of the most important skills. Besides this, we knew almost nothing about Python and camera recognition before starting the project, so learning about new technologies and AI was really useful.

What's next for From Signs to Words

The future updates for the program will be adding words so we can fulfill the whole sign language dictionary and making the interface more esthetic and inclusive.