ASL Translationer

Our Live ASL Translator uses AI to convert sign language to speech in real-time, bridging communication gaps for 11 million deaf individuals and revolutionizing workplace inclusivity.

Comment

Inspiration

Our inspiration came from the stark reality that over 11 million deaf individuals face communication barriers in the workplace. With less than 43% of companies providing reasonable accommodations for deaf employees, we saw an urgent need to bridge this gap using modern technology.

What it does

The ASL Translationer is a real-time American Sign Language to spoken English translator. It captures video frames of ASL gestures, processes them through an AI model to recognize letters, uses ChatGPT to construct coherent sentences, and then employs AWS Polly for natural-sounding speech synthesis.

How we built it

We developed a unique approach using DINO v2's embedding vectors instead of traditional pose estimation. Our tech stack includes:

  • Frontend: React
  • Backend: Flask
  • AI Model: Custom-trained on DINO v2 embeddings
  • Natural Language Processing: ChatGPT for sentence construction
  • Speech Synthesis: AWS Polly

Challenges we ran into

  • Achieving real-time translation while maintaining accuracy
  • Developing an AI model that could accurately interpret ASL without relying on pose estimation
  • Integrating multiple technologies (AI, NLP, speech synthesis) into a seamless pipeline

Accomplishments that we're proud of

  • Creating a working prototype that can translate ASL alphabet in real-time
  • Developing a novel approach using DINO v2 embeddings, setting us apart from existing solutions
  • Building a scalable system with potential for integration into various workplace platforms

What we learned

  • The intricacies of ASL and the challenges in its digital interpretation
  • Advanced AI and machine learning techniques, particularly in computer vision and natural language processing
  • The importance of considering accessibility in technology development

What's next for ASL Translationer

  • Expanding our model to include more of the ASL vocabulary beyond just the alphabet
  • Integrating our technology into video conferencing tools and other workplace platforms
  • Refining the system for even faster and more accurate translations
  • Exploring partnerships with companies to implement the technology in real-world settings
Share this project:

Updates