I originally created this project during my school’s science fair, where the theme was “make everything better with science.” A year ago, I traveled to India and met a boy who could not speak and communicated only through sign language. Because I didn’t know sign language, we couldn’t communicate. That experience made me realize how many people face similar barriers.
This project is my attempt to help people communicate more easily, regardless of disabilities. My project collects images for the letter or sign you want to train, extracts key data points and landmarks using MediaPipe, and trains them using machine learning. During this project, I learned a lot about ML—such as what a neural network is, what backpropagation means, how to build an end-to-end ML pipeline, how to use scikit-learn, and how to use MediaPipe and OpenCV.
One of the main challenges I faced was that my project kept getting deleted on Google Colab due to expired runtime sessions, forcing me to start over multiple times. Eventually, I realized I could use traditional machine learning with MediaPipe instead of deep learning, which made the process much more stable. I hope this project can one day reach a global audience and be distributed to people who need it most.
Built With
- mediapipe
- opencv
- python
- scikit-learn