AI Airpods Hearing: SoundAid

With AI SoundAid, your AirPods and iPhone transform into a powerful and discreet hearing assistant. Giving you a sound boost wherever you are, so you can hear more clearly and comfortably.

Comment

Inspiration

My brother lost his hearing in one ear

A year ago, my brother fainted unexpectedly and smashed his head on the corner of a dresser. He was out for 15 minutes and had to go to the hospital by ambulance. In the hospital they told him he had had a severe concussion. He had to learn to walk again and it damaged his sense of smell permanently. Even stranger: he also lost hearing in his left ear. Not entirely deaf, but severely impaired.

He already owned AirPods Pro (1st gen) and I figured: if these things have beamforming mics and adaptive audio, there must be an app that turns them into a hearing aid? Apple did that for 2nd gen (and since a few weeks the 3rd gen) it should be for any gen, or even non apple earbuds!

What it does

Think of it as Neuralink for your ears. A super human hearing assistant.

The app is for AirPods (normal or Pro only Gen 1) or earbuds. Put them in and turn SoundAid on. You will experience an enhanced experience of sound. Turning on the AI voice focus will filter out the noise and centre the output in human voices. Use the iPhone mic or AirPod mic as audio input.

How we built it

I have no coding knowledge but used cursor + xcode (youtube is my best friend). So prompting about trying everything and restarting about 5 times I was able to make the mock-up of the app and got it working.

Then I sent the app to Alec. He was able to make it look more polished and added a very important feature which was not easy but is a game changer. Thats where the AI in AI SoundAid comes from. We were able to add a live machine learning model that filters out noise and enhances the voice in front of you. We are very happy with the result cause in our opinion it very closely matches Apples audio noise cancellation.

Challenges we ran into

The AirPods Pro have great noise cancellation. Bringing in the audio surrounding you kind of removes that experience. Therefor we had to find a way to add a good filter. We tried alot of different models, we even reached out to Krisp (used on discord for direct audio processing). After lots of iterations we found a great RNN model.

Another challenge: we wanted to market the app as a hearing aid. Since it would help my brother hear beter we figured that would be the way to go. Turns out, apple didn’t think the same. So after going back and forth into app reviews for a month with apple about the medical applications, we chose to pivot. Making the app not a hearing aid but a hearing assistant. Just something used to experience an enhanced environment (maybe apple also didn’t like it cause they were scared we were bringing their technologies to the older gens).

Accomplishments that we're proud of

The RNN we implemented (AI voice filter) sounds and feels quite like the filter apple uses on their voice recordings so we were very happy with the result. We think this makes the app a lot more unique and usable. Imagine using it in a noisy environment like a cafe, you can give the phone to the person you’re talking to and that person can talk right in the mic. Or at a lecture, you’re sitting at the back but there’s people walking in and out etc. Turn on the SoundAid and the AI voice filter and you wont miss a thing.

What we learned

To be able to get the AI voice filter in well we had to learn a lot about audio processing. It’s a whole new world with different types of audio technologies and work arounds that we enjoyed exploring. Also live filtering of audio isn’t the easiest because most models only work with audio files. So if theres another app on audio? We’re all for it!

We also learned to choose app categorization wisely. The medical category for example needs a lot of documentation and certification. Making an app launch very tiresome and slow.

What's next for AI Airpods Hearing: SoundAid

We would like to be able to store the data from the onboarding to personalize the app experience. Basically that would mean everyone has a slightly different looking app based on the answers they gave in the onboarding (like choosing your own story in a book or a game). Like have certain equalizer presets ready to tap for certain situations or even having a different language and UI based on the age and preferences someone puts in.

Furthermore, we hope to help people get a great hearing experience from the app. Hearing aids are around $2000 so its not for everyone. We hope through this app its more accessible to all.

Built With

Share this project:

Updates