Smart-Sign

By combining computer vision and AI to create an ASL translator, we hope to bridge the communication barrier to help those in the ASL community.

Comment 2

Inspiration 💡

There has been more awareness of the difficulties in accessibility for those who use ASL (including but not limited to those who are hard of hearing, deaf, and/or non-speaking). We realize that we take simple things such as ordering food in a drive through, attending class, and going to the grocery store without trouble for granted. Although we cannot fix all accessibility barriers and address the ablelist society we live in, we can at least make a small effort to bring this issue to the forefront of everyone's brains.

What it does ⚙️

By bridging 🌉 the gap between the communication barrier our hack visually renders real-time input to translate ASL into English allowing for those who do not know ASL to communicate.

How we built it 🔨

Using computer vision (OpenCV 🔅) and machine learning (TensorFlow 🤖) we created a training model to compare real-time input 🎥 to recognize ASL signs. We then converted this into an English output using dictionary prediction matching for correcting syntactical and semantic errors.

Challenges we ran into ⚠️

One key challenge we ran into was the difficulty of differentiating between ASL characters with similar hand gestures ✋. Letters like M, N, T and S only differ by the number of finders covering the thumb in a closed fist.

Accomplishments that we're proud of 🌟

Going in, nobody on the team had used TensorFlow or OpenCV for a real project. We are very happy to have learned these tools, completed and submitted a working prototype of our hack project for DeltaHacks VII in just 36 hours ⏰!

What we learned ✏️

This hackathon has taught us how to create datasets 📈 for training machine learning models and how to process images using predictive analysis.

Looking Forward 📌

This project could be improved using a wider dataset than what we could collect in 36 hours to negate the need for a motion capture glove and to expand the software's understanding of ASL beyond the alphabet and into common phrases. A refined ASL translator could improve general accessibility in our society and could lead to a universally 🌌 implemented ASL interpreter.

Built With

Share this project:

Updates