What it does
Medlink is an innovative app that closes the communication gap faced by members of the Deaf and Hard of Hearing communities, who primarily use American Sign Language (ASL) to communicate. MedLink uses Artificial Intelligence and Machine Learning to recognize ASL signs and convert them into text and audio, allowing Deaf individuals to use sign language to communicate with those who are unfamiliar with it. MedLink has the potential to make a positive impact for the approximately 600,000 Deaf individuals in the US. In this pilot version of MedLInk, Deaf individuals can fingerspell (use ASL alphabet signs to spell out words) words into their phone or laptop camera. The app then uses a machine learning model to predict the letter being signed in each frame of the video. It collects the stream of predicted letters and uses a filtering algorithm to extract the text being signed. It has an autocorrect feature to correct any misspelled words, in case the model misses or wrongly classifies a letter. After the user finishes signing their text, the app reads out the text using text-to-speech.
Inspiration
There are approximately 600,000 Deaf people in the US, and more than 1 out of every 500 children is born with hearing loss, according to the National Institute on Deafness and Communication Disorders. These members of the Deaf community primarily use American Sign Language to communicate between themselves, however, with the exception of Deaf individuals and their close friends and family members, most people do not know sign language. This makes it difficult for Deaf individuals to communicate with someone outside of their close circle. Common approaches such as writing out or typing text are slow and interrupt the flow of conversation, making them less than desirable. Through our research, we concluded that there are no released apps available that translate between ASL and English, allowing for natural conversation between a Deaf individual and a hearing individual. As a result, we decided to develop one ourselves, building on our knowledge of machine learning and web app design. We believe that communication is a fundamental human right, and all individuals should be able to effectively and naturally communicate with others in the way they choose, and we sincerely hope that helps members of the Deaf community achieve this goal
Built With
- python
- tensorflow.
