We tried a lot of things but at the end, the ResNet-34 pretrained on the ImageNet dataset performed best for us.
