Under the current wave of globalization, people have the incentive to learn new languages for the ease of communication. Thus, bilingual education is greatly emphasized, and many language learning tools are offered in the market. However, people often forget there is a series of well-developed languages -- the sign languages that are crucial to the deaf community.
What it does
Handlingo is an online platform that allows users to learn and practice different sign languages. Currently, the platform offers the American Sign Language and Italian Sign. Handlingo offers an interactive way of learning. Users can capture your answers with a webcam in real time, to get hands-on practice rather than just memorizing charts. The real time feedback system allows users to learn sign languages in an efficient way. More importantly, it bridges the communication gap that exists when there is a hearing impairment. It is a meaningful digital education tool that brings inclusivity to our community.
How I built it
Challenges I ran into
Connecting the frontend, backend and the model within this short amount of time was quite challenging.
Accomplishments that I'm proud of
We trained two CNN models and were able to reach a relatively high testing accuracy. More importantly, we learnt to collaborate with each other to accomplish the task together.
What I learned
We learned to develop an interactive frontend, specifically on taking images with webcam and sending them to the backend. We also learned to design a machine learning pipeline starting from data collection, model training to building an interactive webpage.
What's next for Handlingo
Implementing the English to Italian translation feature Include more sign languages to the database