-
please sign is detected
-
call sign is detected.
-
hello sign is detected
-
i love you sign is detected
-
yes sign is detected
Inspiration
When a loved one faces the challenge of being unable to speak, this project transforms their silent gestures into audible speech, enabling meaningful communication and connection.
What it does
This project converts sign language gestures into audible speech in real time. It helps individuals who use sign language communicate with those who don't, making interactions more inclusive and accessible.
How we built it
we used:
- XCode IDE -> Integrated Development Environment
- Swift -> Programming Language
- IOS -> Platform for which app is built
- AVFoundation -> Audio Video Framework
- Vision -> Apple framework to identify objects. Eg. Hand, Fingers, and joints
- CreateML IDE -> Integrated Development Environment which helps us create machine learning models like pose, action, text, sound and image classification
- MLHandPoseClassifier -> Framework to add train images, test and export model
Challenges we ran into
Swift is mainly for iOS, so running it on Windows and sharing files was tricky. We solved this by using the Swift toolchain and compiler on Windows, which allowed us to work seamlessly across platforms.
Accomplishments that we're proud of
- Successfully built a real-time sign language to speech converter using Swift.
- Developed an intuitive interface that makes communication more accessible.
- Achieved accurate gesture recognition and smooth speech synthesis in real time.
What we learned
- How to use Swift to build apps with machine learning and real-time features.
- How to train a model to recognize sign language and turn it into speech.
- How to make complex technology easy to use and accessible for everyone.
What's next for Sign Language to Audible Speech
We plan to expand the project by adding support for more sign languages and improving the accuracy of gesture recognition. We also aim to develop a mobile iOS version for easier, on-the-go communication.
