I wanted to make an app to help people, who have trouble hearing, walk on the streets. Initially this app was only aimed at people with hearing disabilities, but I realised that nearly everyone walking/running on the streets is plugged in. So I decided to change the focus of this app to anyone who is walking on the streets, and is unable to hear the sounds on the street.
What it does
This app would warn the user of any honks, sirens or fire alarms by detecting these sounds in their surrounding and vibrating their phones.
How I built it
To build this app I used a cloud database to store sample sounds and used an audio matching sdk to match the sounds in the environment with the ones stored in the database.
Challenges I ran into
The APII was using was mainly focused on music. The biggest challenge I faced was to train the API to compare the warning sounds in the environment with those in the database. To do this I had to first reduce noise in the incoming audio and then process it through the API. Since the API is mainly for music I had to figure out a way for it to recognise sudden sounds like honks.
Accomplishments that I'm proud of
I finally trained the API by making taking the sounds of the sirens, alarms and honks and repeating all of them till I had audio files long enough to be a length of a song. This is not a very elegant solution but I think it is a clever one. Also sufficient for my prototype.
What I learned
To understand how to trick the API I had to dig into it to see how it recognized music. This taught me a lot about sound recognition and also made me realised that it is more complicated than it looks. I would love to learn more about this.
What's next for WarnMe
The next thing I want to do is to allow users to record sounds they want to be warned about. I have an idea on how to do this and I will certainly work on this after the hackathon.