Sign Sense

Inspiration

Sign Sense started with 'Why deaf people are unheeded just because their physical like non-deaf people? In Thailand, handicap person have supported wheel chair and blind people have Tactile pavement but what about deaf? Nothing. Thai deaf experienced a large gap of inequity in society, for example, Education and Job Opportunity.

What it does

Sign Sense is an innovative mobile application designed to facilitate seamless communication between the deaf and non-deaf communities in Thailand. It addresses the communication gap by translating Thai Sign Language (TSL) into spoken language and vice versa. The key features include real-time sign language recognition, translation of sign language into voice, and voice recognition to convert spoken language into TSL. Sign Sense uses image processing technology and a database of sign language building blocks to achieve this, making it possible for both deaf and non-deaf individuals to communicate effectively using their primary language.

How we built it

Even it doesn't connect with voice synthesis API yet We still show the words in text!
Utilizing image processing technology to collect and process images for sign language recognition. Creating a database of sign language building blocks, associating signs with corresponding words.
Implementing Long-Short Term Memory Algorithm to predict sign actions and combine them to form words.
Integrating a speech synthesis system to convert words into voice.
Incorporating natural language processing (NLP) to detect spoken words and their meanings.

Challenges we ran into

Ensuring accurate sign language recognition and translation.
Creating an extensive database of sign language building blocks.
Addressing the unique needs and requirements of the deaf and non-deaf communities.
Collaborating with organizations and collecting data for training the system.
Developing an intuitive user interface for ease of use.

Accomplishments that we're proud of

Successfully developing Sign Sense Demo to bridge communication gaps between the deaf and non-deaf in Thailand.
Achieving an accuracy rate of 70% to 80% in sign language recognition.
Receiving positive feedback from Thai Deaf!
Establishing partnerships with these organizations to access datasets and further develop the technology.

What we learned

The technical challenges of sign language recognition and translation such as, Limit of Deep Learning Algorithm and Limit of some libraries, so, we had to do it on our own!
The potential impact of technology in improving the lives of individuals with communication barriers.

What's next for Sign Sense

Sign Sense want to improve more and more to help deaf
Sign Sense will add more words to database that help deaf communicate with non deaf effectively!
Sign Sense want to do research with deaf people to improve this tool