SunnyGlasses

Sunny Glasses
Product Landing Page
Data Collection

Inspiration

My deaf grandmother has exposed me to many challenges and issues that deaf people face on a daily basis that we typically wouldn't think of. Her and her service dog, Sunny, served as the main inspiration for this project. We wanted to create a piece of technology that could aid deaf people like an automated service dog.

What it does

Our product, Sunny Glasses, is a pair of glasses that can record the user's sign language and then interpret, translate, and speak the sign language into english. Sunny Glasses also can turn speech to text for the user to see as subtitles. This means that a deaf person who only knows sign language can now speak to, and understand speech from someone who does not know sign language.

How we built it

We used computer vision to feed input on a tensor-flow model to help train our algorithm to recognize different terms in sign language. Once our ML recognized a sign we had our software speak the word out loud to essentially speak the sign language. We also incorporated Open Ai's whisper to translate spoken language into subtitles to facilitate two way communication between those who are deaf and those who only know spoken language.

Challenges we ran into

Originally we wanted to run our device through a rasberry pi as the pc with a camera attached to the RPI. However we quickly found out that the RPI did not have enough computational power necessary to facilitate the computer vision and tensor-flow which our device required. We had to redesign our hardware to incorporate a webcam to record the data and feed it into our program.

Accomplishments that we're proud of

Creating our own Machine Learning model to recognize distinct characteristics of different words in sign language and creating our own data set to facilitate this were some difficult achievements we are proud of accomplishing.

What we learned

In completing this project we all learned a lot about how computer vision works, how to use tensor-flow to train your own MLM, how to(and how not to) use a a raspberry pi as your PC, and how to collect and feed our own individualized set of data to a MLM.

What's next for SunnyGlasses

With more time and resources we believe SunnyGlasses could be optimized to a level of performance, compactness, and efficiency which would warrant it being a legitimate product to help those with hearing issues communicate. Whether or not that idea actually comes to fruition may still be up in the air, but it is an exciting feeling to have about a prototype you have created!