Team Straddle_IM23

Inspiration

By 2023, approximately 430 million people, constituting 5% of the global population, experience deafness or hearing impairment. Within this demographic, a significant portion relies on sign language as their primary mode of communication. This circumstance presents a significant communication challenge, as the majority of hearing individuals lack the knowledge of sign language. Consequently, this limitation hampers the social integration, educational opportunities, and employment prospects of the deaf community. These challenges have resulted in a 47% illiteracy rate among prelingually deaf individuals in the UK and a staggering 73% illiteracy rate worldwide.

What it does

The core mission of SignBridge is to serve as a central resource in closing the literacy and educational disparities that affect the deaf community. SignBridge facilitates real-time translations, converting sign language into English and vice versa, employing voice recognition technology to translate spoken language into sign language. This innovation addresses the difficulties faced by illiterate deaf individuals who cannot read written languages. Additionally, the interpreter feature enables seamless real-time communication between deaf and hearing users across various real-life scenarios. Here's a live demonstration to illustrate its functionality.

How we built it

Our team utilised hand landmark detection to locate points on the user's hand to recognise sign language images. Subsequently, we deployed IBM z to train a data intensive deep neural network model to enable the possibility of bridging the communication between deaf people and hearing people through the means of real-time interpretation of sign language.

Challenges we ran into

One of the primary challenges encountered during our development process was the data training required for image processing algorithms, demanding significant computational resources. To address this, we harnessed the processing capabilities of IBM z, significantly enhancing the efficiency of our data-intensive model training, allowing us to work with extensive datasets effectively.

Accomplishments that we're proud of

The current education system prioritises hearing people, with a lack of emphasis for the deaf community, this places the deaf community at a significantly disadvantageous position. The mission of signBridge is to become the pivot to recover the literacy and educational gap for the deaf community.

In aid of this, we have developed a platform to enable the real-time interpretation of sign language to text. Vice versa, we have also implemented the ability to translate from text to sign language to tackle the problem with illiteracy rate. The platform also consists of a comprehensive dictionary designed to assist both the deaf community in learning written languages and hearing individuals in acquiring sign language skills.

What we learned

Through our extensive research efforts, we have gained a profound understanding of the numerous hardships that the deaf community encounters. These challenges encompass a wide range of issues, ranging from the glaring absence of essential resources, such as comprehensive educational materials and accessible communication tools, to the pervasive societal barriers that hinder the full inclusion of deaf individuals in various aspects of daily life. Our research has shed light on the urgent need for innovative solutions, like SignBridge, to address these challenges and empower the deaf community to thrive in a world that should be equally accessible to all.

Dealing with image processing algorithms and real-time interpretation required us to navigate the challenges of handling extensive datasets. Our utilization of IBM z showcased the power of high-performance computing resources in improving the efficiency of data-intensive model training.

What's next for Team Straddle

We will continue to enrich our dictionary by incorporating a more extensive range of phrases, vocabularies, and expressions in sign language. This expansion will enable more effective communication and learning for users. Recognizing the global reach of SignBridge, we aim to include sign languages from different countries. This expansion will make the platform even more inclusive and beneficial to a diverse range of users, allowing individuals from various regions to access and understand sign languages specific to their communities.

In a nutshell, these next steps reflect our commitment to continuous improvement and our dedication to making SignBridge a valuable and inclusive resource for individuals worldwide.

Built With

• dnn
• ibm-z
• jupyter
• python
• tensorflow