SignLingo

Inspiration

The inspiration behind our project stemmed from the desire to create an inclusive learning tool for sign language. Creating accessible learning tools for sign language is not only a matter of inclusivity but also a recognition of the diverse linguistic and cultural needs of the deaf community. Recognizing the significance of sign language in fostering communication for individuals with hearing impairments, we aimed to leverage technology to make the learning process more engaging, accessible, and effective.

What it does

Users can access resources to learn sign language, including visual representations of gestures and associated textual information. The integrated machine learning model enables real-time recognition of hand gestures performed by users. Feedback mechanisms include visual and audio cues along with scores, as suggestions on improvement. Real-time feedback and interactive features enhance user engagement, making the learning process more enjoyable.

How we built it

The project was meticulously crafted using a combination of frontend and backend technologies. React.js was employed for the frontend, ensuring an intuitive and interactive user interface. The machine learning model was integrated into the Flask backend, which served as the intermediary between the frontend and the machine learning component, handling requests, and providing responses.

Challenges we ran into

One of the hurdles faced was fine-tuning the machine learning model to accurately interpret a diverse range of hand gestures. This required extensive data preprocessing, model training, and optimization. Additionally, ensuring a seamless user experience and integrating the machine learning component within the application presented its own set of challenges. In the face of these challenges, our team collaborated, researched intensively, and iteratively refined our project to overcome obstacles.

Accomplishments that we're proud of

Collaborating across disciplines, combining expertise in machine learning, frontend development, and backend, showcased our team's ability to work cohesively and leverage diverse skill sets. We are proud to present a learning tool that not only addresses a crucial aspect of accessibility but also showcases the potential of technology to bridge gaps and create positive change.

What we learned

Working with React.js for frontend development expanded our understanding of component-based UI development, state management, and creating interactive user interfaces. Managing a project from ideation to deployment involved planning, time management, and the ability to adapt to unforeseen challenges.

What's next for SignLingo

Currently our project works with ASL (American Sign Language) and English; as part of the plan to expand our reach we will be working with more sign languages as well as other spoken languages. We also intend to add features which allow users further options to test themselves on the material and track their progress.