Bridging Barriers with AWS & Turkish Sign Language

Leveraging AWS AI/ML services, this project translates spoken language into Turkish Sign Language, enhancing accessibility and inclusion.

Comment

Inspiration

Communication barriers between hearing-impaired individuals and the rest of society have always been a significant challenge. Inspired by the idea of leveraging technology to foster inclusivity, we set out to develop an AI-driven solution. Our initial approach relied on Python-based tools, but AWS services provide a scalable and efficient alternative to enhance accessibility.

What It Does

The system captures spoken words and translates them into Turkish Sign Language. Users can speak naturally, and the system processes the audio, converts it into text, maps it to the appropriate sign language gestures, and displays the corresponding signs in real-time.

Currently, our implementation is based on Python libraries. However, by integrating AWS services, we could further improve accuracy, efficiency, and scalability.

How It Can Be Built with AWS

  • Speech Recognition: Instead of using local speech-to-text models, Amazon Transcribe could handle this step, providing a more accurate and scalable solution.
  • Processing & Mapping: AWS Lambda could be used for real-time text processing and matching words to their corresponding Turkish Sign Language gestures.
  • Sign Language Output: Instead of relying on local storage, Amazon S3 & CloudFront could store and efficiently serve sign language animations/videos.
  • Real-time Communication: With AWS tools, such as DynamoDB for fast retrieval and CloudFront for optimized content delivery, the system could respond in real-time.

Challenges We Ran Into

  • Sign Language Variations: Turkish Sign Language has different gestures depending on context. Finding a standardized mapping for common words remains a challenge.
  • Dataset Limitations: There is a lack of comprehensive Turkish Sign Language datasets, requiring custom data collection efforts.
  • Real-time Performance: Ensuring low latency and fast response times would be easier with AWS Lambda and CloudFront optimizations.

Accomplishments That We're Proud Of

  • Successfully demonstrating how speech-to-text technology can bridge communication gaps.
  • Creating a system that can be further optimized using AWS services.
  • Exploring how cloud-based solutions can provide scalability and accessibility.

What We Learned

  • The importance of real-time processing in accessibility applications.
  • How AWS cloud services like Transcribe, Lambda, and S3 can enhance AI-driven solutions.
  • The significance of inclusive technology for underrepresented communities.

What’s Next?

  • Expanding the Dataset: Creating a larger dataset for Turkish Sign Language.
  • Enhancing UI/UX: Making the interface more user-friendly.
  • Exploring Real-Time Gesture Recognition: Implementing AI models to recognize and generate sign language in real-time.
  • Deploying on AWS: Transitioning to a fully cloud-based solution for better scalability and accessibility.

Built With

Share this project:

Updates