Sign Language To Speech

Inspiration

To bridge gap between sign language and English

What it does

Opens camera, creates skeleton around hand using computer vision and detects what is trying to be said because off a dataset which I have created. Then uses text to speech to say it

How we built it

Using OpenCV and mediapipe I created a application to detect hand movements. Then creates a dataset using my own movements and trained in a neutral network. Then had it predict it in real time

Challenges we ran into

FPS, it very CPU intensive. Had to scale down from torso, face and hands to just one hand because of fps issues.

Accomplishments that we're proud of

Creating a finished product. The computer vision is still quite comprehensive. If I were able to use my GPU I could have been able to created a more smooth and comprehensive system

What we learned

Computer Vision and using it with neutral networks

What's next for Sign Language To Speech

This is a Proof of Concept for a real world application. If given more processing power and a larger dataset. It can be become more comprehensive and clear. It may even be used with some natural language algorithm to be more clear for English speakers. It'll allow the deaf community to express their ideas a lot easier.

Built With

• python