Hands-On Fluency

Sign language education platform

Comment

Inspiration

The inspiration for our project stems from the realization that there is a significant gap in accessible, effective, and engaging resources for learning sign language. With the success of platforms like Duolingo, which have revolutionized language learning, we believe it's time to provide the same level of innovation and accessibility to the deaf community. Learning sign language should be as convenient and enjoyable as learning any other language, and this is where our project comes into play.

What it does

Hands-On Fluency is a web-based online learning platform that assists our users on their journey to discover and learn sign language. We provide video lectures, flashcards, and quizzes to provide an interactive learning environment. Users can create an account with their email, and our website will track their lesson progress, quiz scores, and daily exercise streaks.

How we built it

  • React front-end
  • Flask back-end
  • Jupyter Notebook to train model

Challenges we ran into

  • Handling the user database and encoding their login credentials
  • Implementing real-time hand tracking
  • Creating training data for the CNN
  • File transfer from webpage to the trained model
  • Integrating the frontend and backend files

Accomplishments that we're proud of

  • Creating a neural network to detect sign language
  • Integrating flask back-end with react front-end
  • Solving a real world problem with AI

What we learned

  • How to connect front end and back end
  • How to export machine learning models
  • Coordinated collaboration across multiple developers

What's next for Hands-On Fluency

  • Distribute to educational facilities
  • Further model training to improve accuracy
  • Expand to include other sign languages
  • Implement more features (point system, facial expression detection)
  • Make mobile app version

Built With

Share this project:

Updates