LOCNet

Title

Does combining Discriminative and Generative learning paradigms helps generalization?

Who

Gaurav Gaonkar : ggaonkar, Leyang Hu : hleyang, Shangyang Min : smin27, Zhuoyue Jin : zjin44,

Our project is inspired from two primary studies about texture bias in Discriminative networks and Shape Bias in Generative networks.

Introduction and Related Work

There is a strong evidence in the field of cognitive science [1, 2, 3] showing human biological vision are both texture and shape bias. However when these properties are investigated in state-of-the-art image classification/discriminative models [2], it evident they over rely on texture information as compared to shape. In recent study [4] there is a profound evidence that Generative models has shape bias on level as compared to humans. The idea of the project is to investigate whether applying Discriminative and Generative paradigms simultaneously during classification training helps in learning both the texture and shape information on the level of humans.

Data

We curated a novel dataset with Imagenet images and their corresponding pseudo depth map. For evaluation we are using Out of Distribution data generated using 12 parametric texture noise based filters and 5 shape based filters.

Methodology

We created an encoder-decoder based architecture to perform classification and depth map prediction. More specifically, we are using ResNet50 backbone UNet architecture. Our hypothesis is that for learning depth information, the network will exploit features such as shape, shading which give information about objects 2D or 3D shape leading to better generalization.

Metrics

We are evaluating the classification accuracy of the model on both the in-distribution and out-of-distribution dataset.

Ethics

Why is Deep Learning a good approach to this problem? Ans. We are trying to mitigate the gap between the current SOTA deep learning models and humans.

How are you planning to quantify or measure error or success? What implications does your quantification have? Ans. We are planning to benchmark our model performance on the standard out of distribution dataset.