Sign Learn

An AI-powered web app that teaches ASL through real-time sign recognition, webcam-based practice, and instant feedback—making ASL learning accessible, fun, and inclusive.“Sign. Learn. Connect.”

Comment

Inspiration

This project was inspired by a desire to bridge the communication gap between the Deaf and hearing communities. Learning American Sign Language (ASL) can be intimidating or inaccessible for many, especially without hands-on guidance. We wanted to build something that makes ASL learning approachable, engaging, and truly interactive—with no special equipment or prior experience required.

What We Learned

  • The fundamentals of computer vision and how it can be applied to gesture recognition
  • How machine learning models like Random Forest can classify hand signs effectively
  • Real-time webcam integration using OpenCV and MediaPipe
  • Building accessible web apps that are intuitive and inclusive

How We Built It

  • Hand Detection: Used Google’s MediaPipe to track hand landmarks from the webcam
  • Sign Classification: Trained a Random Forest model to recognize ASL alphabet signs based on hand landmark positions
  • Interface: Developed a Python-based GUI for desktop, and are now working on converting it to a browser-based app for broader accessibility
  • Feedback Loop: Implemented real-time prediction display and user prompts to help learners practice and improve

Challenges We Faced

  • Ensuring high accuracy across diverse lighting conditions and hand sizes
  • Distinguishing similar-looking ASL signs (like G and H)
  • Creating a responsive, real-time system without lag
  • Designing a user-friendly interface that works smoothly on the web

Built With

Share this project:

Updates