HackSMU VII: Babel Elephant

Inspiration

We were inspired by the iMasons challenge statement, which we are competing for, and the fictional organic translator called the Babel Fish from The Hitchhiker's Guide to the Galaxy (1978).

Unfortunately, the Babel Fish's capabilities are purely within the realm of fiction, owing to difficulties in real-time translation such as having to wait for the rest of the sentence. However, while efficiently removing noise from an audio file without degradation of the audio remains an elusive goal, it is at least within the realm of possibility, and perhaps might get humanity slightly closer to an elephant translator.

What it does

Our project builds a neural network to parse an audio file, and at every frequency and timestamp, determine whether noise exists and remove it if so.

How we built it

Following the approach outlined in existing research, we parse a WAV file into an audio tensor which is then converted into a spectrogram tensor using a short-time Fourier transform. After preprocessing the spectrogram, we use a convolutional neural network to identify noise to remove. We can then remove the noise by adjusting the magnitude of the spectrogram, and use the phase of the original to perform an inverse STFT to convert back into an audio file.

Challenges we ran into

While the existing research uses MFCC features to identify characteristic features of elephant rumbles to find their timestamps, our biggest problem is that we need more specific information than timestamps in order to clean up overlapping noise without degrading the quality of the elephant call itself.

Accomplishments that we're proud of

Considering the short time constraint of at most 24 hours, our team of 3 people, and our combined relative lack of subject-matter expertise, we're very proud of being able to train a model and demonstrate our program.

What we learned

None of us had ever worked with machine learning on audio data before, but we were successfully able to learn as we went how to transform audio files into different kinds of tensors that we can work with and back into audio files again with various types of Fourier transform functions in Tensorflow. We also didn't know anything about elephant rumbles. We were able to quickly learn about existing work in the area, and were also fascinated by further areas of research from the biological perspective such as the rich information believed to be contained in elephant rumbles that are still unknown.

What's next

Our approach was binary in regards to the existence of noise or not. In reality, it may be the case that in a given frequency and time, a true elephant rumble may be lost due to overlapping noise. An interesting future extension would be how to extend this solution use harmonics to reconstruct the missing fundamental or vice versa, similar to what human ears do. Another further area of research would be how to construct an appropriate test function in order to determine how effective the network was at removing noise.

Built With

• numpy
• pandas
• python
• scikit-image
• scikit-learn
• tensorflow