There have been hackathon projects that are meant to help the deaf before, but none seem to capitalize on one of the deaf's most powerful aspects - the ability to read written language. Most deaf people are able to read and comprehend written English, so we developed Hubert the Deaf Helper to aid them as they live out their lives. When going out in public places, Hubert will allow deaf people to see what words others are directing towards them using a companion app. We hope that Hubert will prove useful to deaf people that want to go out in public without a worry.
What it does
Hubert includes three microphones attached to the end of a fanny pack ('Hubert Holster') that listen for speaking voices. The audio is transferred through the Google Cloud to the companion app, that translates the speech to text and displays it on the screen. Hubert will allow deaf people to know what people are saying to them, simultaneously improving their reading skills by exposing them to other people talking.
How we built it
We used a Particle Argon to receive the input of three microphones connected by wires. We used the Google Cloud to transmit data from the Particle Argon to the Android App, which we developed using Android Studio. We used the Google Speech-To-Text Artificial Intelligence models to convert the audio to text and display it on the app.
Challenges we ran into
- We were trying to figure out how to send the data from Hubert to the Android App. We ran into troubles with Bluetooth LE as it did not send data packets as large and quick as we needed it to. We eventually settled on manipulating the data to send it through the Google Cloud, which required its own troubleshooting.
- The wiring would act up at times while we were setting up the microphones and they would stop receiving audio. We had to completely change the orientation of the wires on the breadboard so we could tape them down and keep them safe to carry around.
Accomplishments that we're proud of
- We worked hard and it's almost working. Temporarily worked and then broke.
What we learned
- From the project we learned how to use Big Data in the Google Cloud to transmit messages between devices. We also had to learn how to use the Google Speech-To-Text API and the Google Cloud Console to manage the project.
What's next for Hubert - The Deaf Helper
After growing past his humble beginnings at hackPHS, Hubert will only be improved. With a greater amount of time and resources, we will be able to make Hubert easier to carry, optimize the speech recognition and support multiple languages, and test it with deaf people to get some legitimate feedback.