Through the Centuries: Classifying Genres of Art

From Modernist interpretations to even heralded classics of the Renaissance, art genres have always been difficult to distinguish. By utilizing deep learning technologies, we can accomplish just this!

Comment

Final

Introduction

Art can often be a fickle topic — its intrinsic value, restricted to canvas and oil-based paint, is rather opaque. Ranging from Modernist interpretations to even heralded classics of the Renaissance, art has always been difficult to decipher. Its universal critique, across ages, is evidence of this fact. Especially to the untrained eye, potentially those of a Computer Science enthusiast, it is rather difficult to discern genres and movements spanning decades and even centuries, even despite underlying appreciation and admiration.

Imagine a scenario: it’s another post-graduation alumni meet up and this time it’s hosted at the local art museum. Your last encounter with art was “Art History: Impressionism in the early 1900s” with Professor Schumer, so you’re a tad rusty. If only there was a simple way to aid you with navigating through all of the paintings.

Utilizing deep learning technologies, we can accomplish just this. Without the need for years of in-depth analysis and research, we can expand knowledge to a wider range of individuals, increasing accessibility to the arts that so people lack.

Related Work

Previous individuals have used the exact same dataset to create a Convolutional Neural Network that would predict the artist based on the training data of a set of paintings. The model took the 50 most influential artists of all time and used colors and geometric patterns to predict who painted what painting. We realized that there was more we could do with this dataset as it gave insight into the years in which the artists lived and the genre under which the painting is classified.

Namely, in this blog, Abishek explains his process in his attempt to use CNN to match paintings up with their artist. His process begins as any ML model would with obtaining training data. Then he augments and transforms his data slightly to create more appropriate training data. After convolution and two dense layers, he is able to achieve a training set accuracy of 87% and a testing accuracy of 77%.

Linked here is also some other inspiration we had: DeepArtist : Identify Artist from Art & Impressionist Classifier.

Our Data

The dataset we're using is from Kaggle, scraped from artchallenge.ru at the end of February 2019 by the user ICARO. The dataset itself contains 1000+ paintings from 50 artists. This dataset contains three files: artists.csv, images.zip, and resized.zip.

Linked here is the dataset.

Methodology

  • Considering multiple options, we believe a ResNet-50 architecture is most optimal due to previous image-classifying projects also using ResNet.
  • Additionally, we plan to utilize data augmentation, but we’re currently unclear as to what is the most efficient manner to do so (i.e Zoom, Rotate, etc.)

  • After multiple layers of convolution, we plan on flattening, batch normalization, and many dense layers that primarily use LeakyReLU as the activation function, and finally ending on softmax to create a confusion matrix on our output.

    • For our optimizer, we anticipate using Adam.
    • For our loss function, we anticipate using cross-entropy loss.
    • For our accuracy metric, we anticipate using categorical accuracy.
    • Hyperparameters — such as learning rate, batch size, epochs etc. — will be determined through trial and error once the model is complete. Our success metric will be the model’s accuracy.

Accomplishments that we're proud of

Following training, we intend to utilize categorical accuracy to validate against the testing dataset. If our CNN is able to classify to the genre appropriately, we deem that a “successful” classification and “unsuccessful” otherwise.

  • Base goal: achieve 40% categorical accuracy on training evaluation and 20% on test evaluation.
  • Target goal: achieve 60% categorical accuracy on training evaluation and 40% on test evaluation.
  • Reach goal: achieve 80% categorical accuracy on training the evaluation and 60% on test evaluation.

Ethics

Why is Deep Learning a good approach to this problem?

  • Genres of art aren’t confined to strict lines, making the classification of art almost art in itself. Dependent on time, country, and artist's style, each painting is unique but is able to fall under a category. Deep Learning, now, can offer a different opinion based on numbers and data. It can take a more in-depth view of a painting and based on strokes, colors, and style it can attempt to classify the image, and we can see if there is something quantitative that allows art enthusiasts to be able to glance at a painting and classify it.

What is your dataset? Are there any concerns about how it was collected, or labeled? Is it representative? What kind of underlying historical or societal biases might it contain?

  • Our dataset was scraped from artchallenge.ru, a game where you can guess the artist from different time periods. There aren’t any concerns about how the paintings were collected, but there are some questions as to how it was labeled as some genres of paintings are slightly ambiguous. It isn’t fully representative of all styles and time periods as it has little to no modern paintings and only very famous paintings. Since they are simply pieces of art that are labeled there isn’t any historical or societal biases in the dataset.

Divison of Labor

  • Kevin

    • Project Proposal (Shared)
    • Prepossessing
      • Remove redundant genres
    • Model
      • Create CNN model
      • Final Report, Poster, Video (Shared)

  • Sid

    • Project Proposal (Shared)
    • Prepossessing
      • Remove paintings classified with multiple genres
    • Model
      • Create visualizations (graphs, confusion matrices etc.)
      • Final Report, Poster, Video (Shared)

Built With

Share this project:

Updates