SignLingo

A sign language interpreter that uses your computer's webcam to decipher you hand signs. The application has been gamified to give points for each correct sign so that you can have fun while learning!

Comment

SignLingo

SignLingo is a web application that utilizes computer vision to recognize hand signs using the MediaPipe library. The application allows users to interact with sign language gestures, providing a video feed with real-time hand sign detection.

Inspiration

The inspiration behind SignLingo comes from the desire to create an engaging and educational tool for learning sign language. Recognizing the importance of accessibility and inclusion, the goal is to provide a fun way for users to practice and improve their sign language skills.

Features

  • Real-time hand sign detection using the MediaPipe library.
  • Dynamic scoring system to track the user's performance in recognizing signs.
  • Interactive web interface with a video feed and a changing set of hand signs.

What We Learned

During the development of SignLingo, I gained valuable insights into the following areas:

  • Computer Vision: Explored and implemented real-time hand sign detection using the MediaPipe library, enhancing my understanding of computer vision techniques.

  • Web Development with Flask: Learned how to integrate a computer vision application into a web interface using the Flask framework, enabling users to interact with the system through their browsers.

  • User Interface Design: Developed an intuitive and interactive user interface to enhance the overall user experience.

How We Built This Project

SignLingo is built using the following technologies:

  • Python: Leveraged the power of Python for implementing the backend logic, including hand sign recognition.

  • Flask: Created a web application using the Flask framework to provide a user-friendly interface for interacting with the hand sign detection system.

  • MediaPipe: Integrated the MediaPipe library to perform real-time hand landmark detection, forming the basis for hand sign recognition.

Challenges Faced

Building SignLingo posed several challenges, including:

  • Integration of Computer Vision with Web Development: Combining computer vision functionalities with web development required thoughtful integration to ensure seamless user experience and real-time updates.

  • Using Flask for the First Time: Working with Flask was very challenging as it was not like any other frameworks we had used before and ran into many problems with regards to the integration of the python and frontend frameworks.

  • Scoring System Design: Designing a scoring system that accurately reflects the user's performance and dynamically updates in real-time was a unique challenge.

Future Plans

As the application currently only recognises 3 hand signs, we plan to improve this by allowing the application to recognise more signs. This will be done by training a DL model with training data collected by us using the webcam in order to optimise its abilities.

Built With

Share this project:

Updates