GenAI-powered Global Sign Language Translator

text input
voice input
audio file input
Architectural Diagrams

Inspiration

One day, I watched a movie trailer “The Way We Talk” that explored the perspectives of deaf-mute individuals regarding cochlear implants. I realized that society often hopes these individuals will become "normal" by gaining the ability to hear and speak. However, it overlooks those who prefer to communicate through sign language.

Inspired by this story, we launched this project. We developed a sign language translator using the GCP Platform, translation API, and Vertex AI. Our goal is to help deaf individuals engage with the world in a way that feels most comfortable for them.

What it does

“GenAI-powered Global Sign Language Translator” provides main functions: An AI-assisted communication system that can automatically convert spoken language or text into virtual human sign language animation, aiming to break the communication barriers between hearing-impaired people and others

This technology promotes accessible communication, enabling individuals with hearing impairments to easily understand and engage in conversations.

By efficiently generating ASL animations, it breaks the limitations of traditional tools and enhances social inclusivity, providing valuable support for the hearing-impaired community.

How we built it

The following shows a Simplified Architectural Diagrams:

Challenges we ran into

Speech Recognition Accuracy Issues Complexity of ASL Gloss Conversion

Technical Challenges in AI Animation Generation

Accomplishments that we're proud of

Developed a solution that makes communication more accessible for the deaf community Created a system capable of handling multiple input methods to accommodate different user needs