Multi Sign Language : Speech and Text Converter

Inspiration

The inspiration for the "Multi Sign Language: Speech and Text Converter" project came from the desire to bridge communication gaps for individuals with visual and speech impairments. Observing the challenges faced by those who use sign language to communicate, I aimed to create a tool that could seamlessly convert American Sign Language (ASL), British Sign Language (BSL), Spanish Sign Language (SSL), and Indian Sign Language (ISL) into text and speech, making interactions more accessible and inclusive.

What it does

The project translates sign language gestures into both text and speech, facilitating communication between users and their surroundings. It supports multiple sign languages, including ASL, BSL, SSL, and ISL, providing an effective means for individuals with hearing and speech disabilities to communicate with others who may not be familiar with sign language.

How we built it

The project was developed using a combination of technologies:

scikit-learn: For building and evaluating the Random Forest Classifier model.
NumPy: To handle numerical operations and data manipulation.
OpenCV: For image processing and capturing gesture data from video input.
Mediapipe: To detect hand landmarks and extract gesture features from images.
pyttsx3: To convert text into speech.

Challenges I ran into

Data Collection: Gathering a diverse and representative dataset of sign language gestures was challenging. Ensuring the data was accurate and comprehensive required significant effort.
Gesture Recognition Accuracy: Achieving high accuracy in gesture recognition involved fine-tuning the model and addressing variability in individual signing styles and environmental conditions.
Integration: Combining various technologies (OpenCV, Mediapipe, scikit-learn) into a unified system posed technical challenges, particularly in ensuring smooth interaction between components.

Accomplishments that I am proud of

High Accuracy: Achieved a 95% accuracy rate in converting ASL, BSL, SSL, and ISL into text and speech.
Comprehensive Support: Successfully integrated support for multiple sign languages into a single application.
Accessibility Impact: Developed a tool that significantly enhances communication options between visually impaired and speech-impaired individuals.

What I learned

Model Training and Evaluation: Gained experience in training machine learning models and evaluating their performance, specifically using scikit-learn.
Gesture Recognition Techniques: Learned about effective methods for gesture recognition using Mediapipe and OpenCV.
Technology Integration: Acquired insights into integrating different technologies and libraries to build a cohesive and functional application.

What's next for Multi Sign Language: Speech and Text Converter

Expand Language Support: Include additional sign languages to further broaden the application's usability.
Enhance Accuracy: Continue refining the model to improve accuracy and handle a wider range of signing styles and environments.
User Interface Improvements: Develop a more intuitive and user-friendly interface to enhance the overall user experience.
Community Engagement: Engage with communities of sign language users to gather feedback and continuously improve the tool based on real-world needs and experiences.