Aural

Design for the hard of hearing and ASL learner. Enrich the quality of daily lives with Aural.

Comment

Introduction

Prior to the COVID-19 pandemic, hearing loss was already linked to social isolation throughout life, and research shows it can produce emotions such as stress and anger. However, COVID-19 has increased and highlighted existing inequities for the hard-of-hearing inbuilt and social environments. This affects nearly one out of five people (1.5 billion people) who experienced hearing loss in 2019.

Facial masks, an essential tool used to slow the spread of COVID-19, can decrease speech recognition for some people with hearing loss. They can obstruct facial expressions and lip movement, which are critical sources of non-verbal information. In a recent Hearing Health Foundation survey, a reported 85 percent of hearing-impaired individuals experienced difficulty reading lips covered by face masks during the pandemic. While clear face masks can reduce lip-reading challenges, access and usage can vary by setting, making it harder to navigate unfamiliar social contexts and communicate effectively.

Even accessing health care has become more difficult for hard-of-hearing individuals during the pandemic. Telemedicine platforms and hospitals with strict visitor restrictions do not always provide translation services for people with hearing loss.

Aural is a mobile hearing assistant application that records any surrounding audio and provides you with the live transcript and translation, as well as a live ASL caption. To hear with your eyes.

Purpose & Motivation

We were inspired by our observation and research with hard of hearing community and their challenges during day-to-day communication.

Our goal is to improve the communication between the hearing and hard of hearing people, we noticed the lack of support for English as their second languages user in live transcript during daily life. We would also like to create a bridge for anyone who's interested in ASL or someone at their beginning journey of learning ASL. During our research, we came to the conclusion that sometimes it is easier to quickly sketch their mind rather than type their idea out. With this information in mind, we created Aural.

How does the application work?

Aural is a mobile hearing assistant application that records any surrounding audio and provides you with the live transcript and translation, as well as a live ASL caption. The 3 main features are live transcription with language translation, transcript of previously recorded audio, and open camera ASL caption for anyone who would like to learn about ASL.

Notepad allows users to better communicate, such as drawing a map when asking for directions, etc.

How was the application developed?

Aural was made with Flutter, a cross-platform framework so that it can cater to a large audience. Tensorflow model was trained for ASL translation and pub. dev packages were used to integrate audio to text and language translation.

Difficulties & Challenges faced during the design and/or development process.

With the design, we would like to keep accessibility in mind. We offered to increase and decrease the font size in the transcript section. The overall style of the application we choose to use legible fonts such as Roboto and Montserrat, contrast colors, proper hierarchy, and text sizes to make for a better reading experience. The dark blue color scheme is to make it more colorblind-friendly.

What did we learn:

  • We learned a lot about different packages, the implementation of the assembly API, and other stuff.
  • We learned how to organize our time effectively.
  • We learned how to have good teamwork by communicating effectively using discord.

What's next for Aural:

  • Making it more accessible by improving UI based on user feedback.
  • Adding emojis and prebuilt text choices for making communications between the deaf person and the person next to him/her easier.
  • Publish our App toGoogle Play Store.
  • Fix bugs and depend less on less reliable third-party libraries, which might cause potential security issues and bugs.

Built With

Share this project:

Updates