ASL Amigo

Armature View of L
Mesh View of L
Rendered View of L

Inspiration:

Our ASL translator, ASL Amigo, was inspired by a partially deaf team member. Witnessing the challenges he faced in communication sparked the idea to create a tool that could assist not only him but also others in the deaf community.

What it does:

ASL Amigo is capable of translating American Sign Language (ASL) hand gestures into text in real-time. It bridges the communication gap between individuals who use ASL and those who do not understand the language, providing a means for seamless interaction.

How we built it:

We utilized synthetic data generated with a 3D software called Blender to rapidly create samples for training our Teachable Machine model. This approach allowed us to include various variations of images, such as different backgrounds, lighting conditions, and accessories. We integrated this data into our model and employed several different libraries to develop ASL Amigo, ensuring its accuracy and accessibility as an application.

Challenges we ran into:

Throughout the development process, we encountered several challenges. Training the model to perfection was a significant hurdle, requiring meticulous fine-tuning and optimization to achieve accurate translations. Additionally, making the application more accessible presented its own set of obstacles, as we aimed to cater to a diverse range of users with varying needs and preferences. Working with multiple libraries and integrating them seamlessly into our product also posed a challenge that we had to overcome.

Accomplishments that we're proud of:

We are proud to have developed ASL Amigo, a tool that addresses a real-world need and has the potential to significantly improve communication for individuals who use ASL. Achieving accurate translations and creating a user-friendly interface were significant accomplishments for our team. Moreover, overcoming the challenges we faced throughout the development process showcases our determination and problem-solving skills.

What we learned:

Developing ASL Amigo taught us valuable lessons in machine learning, accessibility, and user interface design. We gained insights into the complexities of training models with synthetic data and the importance of creating inclusive applications that cater to diverse user needs. Working with various libraries and technologies also expanded our knowledge and skill set.

What's next for ASL Amigo:

Looking ahead, we have ambitious plans for ASL Amigo. We aim to enhance its capabilities by implementing fluent phrase translation, allowing for more natural and comprehensive communication. Additionally, we plan to refine the user interface to make it even more intuitive and compatible with different devices and platforms. Our goal is to continue improving ASL Amigo to better serve the deaf community and facilitate meaningful communication for all.