Cross-Species Translation (Dog barks)

Inspiration

Successful “translation” models have numerous applications in conservation and wildlife monitoring, comparative cognition research, and even human-animal interaction.

What it does

Our model takes 4 second long dog bark audios as input and makes predictions for the context (aggression, play, or contact), breed, and dog identity (name).

How we built it

We first used k-means clustering to segment four second barks from the original audio files as a form of preprocessing. We then trained three CNNs each with four convolution layers to perform multi-label classification of individual barks based on context, breed, and dog identity.

Challenges we ran into

A challenged we faced was the fact that the dataset had clear majority and minority classes, with some individual dogs having hundreds of barks and others having less than one hundred. To prevent the model from simply learning to predict the majority classes, we penalized it more for getting the minority classes wrong with class weights, resulting in a confusion matrix that looks much more balanced.

Accomplishments that we're proud of

We are most satisfied with our accuracies, with 83% for predicting the dog breed, 72% for context, and 67% for identity. These accuracies also make intuitive sense, since one could guess that it is easier to distinguish dog barks based on breed compared to dog identity.

What we learned

When approaching similar projects, we would recommend starting very early and prioritizing modularity. A project like this that can easily be extended to other datasets and should be eventually able to generalize across datasets should be highly modular to make transitions easy and organized.

What's next for Cross-Species Translation

If we were to continue with this project, we would first create models for other species, as the dataset we found (Earth Species Project) has audio recordings of many different species, and attempt to create a model that can generalize across species. It could also be interesting to interpret the model and attempt to understand what patterns specifically the model finds in order to make generalizations about animal communication.