Classifying Marine Mammal Vocals w/ Machine Learning

The goal of this project is to explore marine animal vocalization classification using machine learning and Dolby IO media processing. I will be implementing a convolutional neural network.

Comment

Inspiration

"Because water is denser than air, sound travels very efficiently underwater. Sounds from some species of marine life and human activity can be heard many miles away and, in some cases, across oceans.

Passive acoustic instruments record these sounds in the ocean. There are some hydrophones that generate up to 24 terabytes a year! "e.g. Big Data"

This data provides valuable information that helps government agencies and industries understand and reduce the impacts of noise on ocean life.

By listening to sensitive underwater environments with passive acoustic monitoring tools, we can learn more about migration patterns, animal behavior, and communication." quoted from NOAA

What it does

The trained Convolution Neural Network model can be used to run inferences on unseen marine mammal audio in an effort to classify its species.

How we built it

  • Web Scraped audio from Watkins Whale Database
  • Dolby IO media processing *Tensorflow, Keras
  • Python, Librosa, Essentia, Pydub, FFmpeg

Challenges we ran into

Web scraping and becoming familiar with Dolby IO API to process batches of audio "training data" of over 1000 audio files.

Accomplishments that we're proud of

Augmented audio files to synthesize more data: +/- 2dB, +/- 2 semitones and added random noise

The train CNN resulted in over 97% accuracy on the unseen test set.

What we learned

Audio preprocessing for ML, interacting with Dolby IO API, Training a model on MFCCs

What's next for Classifying Marine Mammals Vocals w/ Machine Learning

Implement Real-Time analysis using the audio library Essentia

  • add more marine mammals
  • introduce human-generated sounds like vessels, cargo ships, etc.

Built With

Share this project:

Updates