SIGNTOR

Singtor aims at tackling issues faced by people using sign language, a hand gesture recognizer using OpenCV and python. MediaPipe and Tensorflow framework for the detection and gesture recognition.

Comment

Inspiration

The inspiration behind Signtor is to provide a communication solution for individuals with speech and hearing impairments. The project was developed during the global lockdown period in 2020 and 2021 when it was realized that practical sign language usage was not feasible through various usually used platforms. As an impaired person, one can often struggle to communicate in real-time, negatively impacting daily life. Signtor aims to solve this problem by using hand gestures as a form of communication that can be recognized in real-time using machine learning and computer vision technologies. The project was developed to provide hands-on experience in developing an app for people with speech and hearing impairments and to provide a practical solution to a real-world problem.

What it does

Singtor is a gesture recognition application that uses the latest machine learning technology to recognize hand gestures in real time, providing users with a seamless communication experience. Using MediaPipe, an open-source, cross-platform framework, Signtor detects hand and critical points with high accuracy, which are then fed into a pre-trained gesture recognition network built on TensorFlow, a popular machine learning library.

How we built it

•Import necessary packages. • Initialize models. • Read frames from a webcam. • Detect hand key points. • Recognize hand gestures.

Challenges we ran into

As with any complex software project, building poses several challenges. Here are a few of the main ones we ran into:

  1. Scalability: As the platform grew in popularity, we had to continually optimize our infrastructure to ensure it could handle the increased traffic and demand. This involved a lot of performance tuning and load testing to identify bottlenecks and improve system efficiency.
  2. Speech recognition accuracy: One of the core features of Signtor is its ability to transcribe speech to text in real time. Achieving high accuracy in this area was a significant challenge, as it required us to develop and fine-tune our speech recognition algorithms to work across various languages and accents.
  3. User experience: We wanted to make sure that using Signtor was as easy and intuitive as possible for our users, which meant investing a lot of time and effort into designing a clean and user-friendly interface and developing robust and reliable backend systems to support it.
  4. Data privacy and security: Given the sensitive nature of the conversations on Signtor, we needed to build robust security and privacy features into the platform from the ground up. This involved everything from implementing end-to-end encryption to building secure authentication and access control mechanisms. ## Accomplishments that we're proud of

Signtor has achieved several accomplishments since its inception, including: Impacting rural healthcare: Signtor has helped bridge the gap in healthcare access in rural areas of India by enabling telemedicine consultations between patients and doctors.

What we learned

Signtor is likely to involve a wide range of challenges and learning experiences. Some possible lessons that the Signtor team may have learned during the development process include:

  1. The importance of collaboration and effective communication: Building a project like Signtor would require close cooperation between people from different backgrounds and areas of expertise. The team would need to communicate effectively to ensure that everyone was working toward the same goals and that each team member's work was integrated smoothly with the work of others.
  2. The value of persistence and flexibility: Developing a project like Signtor would likely involve encountering unexpected problems and setbacks. The team would need to persist in facing these challenges, finding creative solutions and remaining flexible in their approach as needed. ## What's next for SIGNTOR

In the future, we plan to continue improving Signtor by adding new features and expanding its capabilities. Some of the areas we are exploring for future development include:

  1. Natural Language Processing: We plan to incorporate more advanced natural language processing capabilities into Signtor to enable more natural and intuitive interactions between users and the system.
  2. Speech Recognition: We are exploring ways to improve signtor speech recognition capabilities, such as by using deep learning algorithms to improve accuracy and expand the number of supported languages.
  3. Text-to-Speech: We plan to add text-to-speech capabilities to signtor to enable users to convert written text into high-quality audio content.
  4. Accessibility: We believe that Signtor has the potential to be a powerful tool for creating content that is accessible to people with disabilities. To this end, we are exploring ways to make the system more user-friendly for people with disabilities by incorporating features like speech-to-text and text-to-speech.

Built With

Share this project:

Updates