hearo

hearo - let hearo be your hearing power
hearo landing page
hearo speech to text stream

Inspiration

Our team members have experienced complications when communicating with elderly loved ones troubled with hearing impairments. Specifically, the need for another translator when communication is lost. With this in mind, we aimed to provide a simple solution that tackles the issues of hearing disturbances with the use of hearo.

What it does

hearo is an accessibility tool that promotes independence and acts as a backup solution for potential hearing aid malfunctions. In the case an individual encounters disturbances with their hearing aid device, the hearo app can be used to offset communication via a speech to text stream such that the individual can use other means of communication without the need of hearing.

How we built it

The hearo app was first designed and outlined using figma. The front end was built using the React framework coupled with JavaScript, HTML, and CSS. Furthermore, we implemented Node.js, Twilio, and Assembly AI to implement the speech to text features.

Challenges we ran into

The most difficult challenge our team faced was implementing the transcript onto the React page. AssemblyAI allowed us to transcribe the phone calls from Twilio in real-time but the original implementation was for the transcription to be printed in the terminal. Our team learned a lot about WebSockets to troubleshoot this issue. It was also fun taking on the challenges of incorporating Twilio and Assembly AI packages provided.

Accomplishments that we're proud of

We are most proud of our production of an app that spoke dearly to the challenges we faced commonly in our life experiences.

What we learned

We learned that AI applications lend themselves as wonderful tools that can be further utilized to fill in gaps in solving specific solutions. It was a wonderful experience learning to incorporate these commands and build them into our project.

What's next for hearo

When brainstorming this project, we had a variety of ideas we wished to implement but unfortunately ran out of time. Firstly, we wish to integrate a video chat feature where users can see each other in real time and have their conversations be transcribed for them to see on the side. In addition, we hope to develop a phone app for iPhone and Android users, making hearo much more accessible for everyone. hearo also looks forward to further implementations that can further close the accessibility gap in individuals with hearing impairments. It looks to further tie solutions to those in different situations, and possibly incorporate features that could also work around language barriers.