Food for Thought — Paper Tuesday

Food for Thought — Paper Tuesday

Use Shape-biased Data Improves Accuracy and Robustness

Every Tuesday I highlight an interesting paper that I came across in research or work. I hope that my review can help you get the juiciest part of the paper under 2 minutes!

Basic Ideas

It is widely accepted that CNNs learn images by extracting shape features like curves and edges. However, a group of researchers from University of Tuebingen and Edinburgh challenged this belief in their ICLR 2019 paper ImageNet-Trained CNNs Are Biased Towards Texture; Increasing Shape Bias Improves Accuracy and Robustness .

Here’s the link: https://openreview.net/pdf?id=Bygh9j09KX

By cleverly crafting several experiments, the researchers demonstrated CNNs are more biased toward image textures than people would expect. From that, they further found that shape-enhanced dataset can serve as an effective data augmentation method that improves model accuracy and robustness.

The researchers argued that CNNs are heavily biased toward local features, perhaps due to the small perception field of convolution filters. Their argument is supported by CNN’s surprisingly low performance on texture-free images demonstrated in the image below

As demonstrated in the figure, all major-league architectures like AlexNet, GoogleNet, and VGG16 experience significant performance drops when texture information is removed (silhouette and edge). Meanwhile, CNNs yield high confidence even when shape information is removed as long as texture is present (texture).

Results

In order to further test their hypothesis, the researchers generated a new dataset called Stylized-ImageNet (SIN), whose images’ local texture features are replaced by uninformative random features.

If CNNs are biased toward local texture features, we would expect CNNs trained on the original ImageNet dataset to perform poorly on SIN. This is indeed the case, as demonstrated in the following table

Okay, now we are convinced that CNNs are biased toward local textures. But how can we use this inforamtion to our advantage? The researchers demonstrated that models jointly trained on SIN and IN is more robust to image distortion (noise, cropping, filtering…) and achieve outstanding accuracies in image classification and object detection.

Some Thoughts

For a long time I was convinced that CNNs are capable of image classification because of its powerful edge detectors. This paper opened a new door for us — there are some many handwaving explaination and understanding of neural networks. There are still a lot of theoretical work ahead to understand even one of the simplest forms of neural networks!