ASL Computer Recognition

Inspiration

My main inspiration came from watching engineering YouTubers use camera tracking and recognition within their videos. It made me want to step out of my comfort zone to learn about creating virtual bones that a computer will be able to train to read and translate from the American Sign Language (ASL) to English.

What it does

The program is able to track your each part of your hand and it was trained to understand which ASL hand positions correlated with which English letter/number.

How we built it

I used python to code everything in the project. The main packages that made this project possible were "mediapipe" and "cvzone" as they handle every part of the finger tracking.

Challenges we ran into

A lot of the challenges came from using a newer version of Python which made mediapipe fail in some scripts. Having to work around that by using a similar package, cvzone, was an arduous process.

Accomplishments that we're proud of

I am proud to say that this is my first major project that I have worked from start to completion. As well as, creating a website for the program to be hosted on.

What we learned

I learned how computers are able to synthetically "see" through creating virtual bones that would follow my hand movements/positioning.

What's next for ASL Computer Recognition

The next step-up would be to introduce hand movements to be able to form long words/sentences instead of only stationary hand positioning.

Built With

• av
• cvzone
• mediapipe
• opencv
• python
• scikit-learn
• streamlit
• twilio