The issue is that the networks are using a trivial, nonessential stimulus feature to base classification decisions on. It's like basing the decision on a very small-eigenvalue feature eigenvector. That could be a training set issue, or it could be something about the network structure.

The article was kind of interesting to me because it reveals that networks are probably sometimes making decisions on highly discriminating but non-essential stimulus features.

It's like the networks might have a high success rate with large number of replicated examples, over sample size, but not over a large number of distinct examples.

They're overfitting, but in a way that isn't immediately obvious because the test stimuli tend to be limited.

My guess is the answer is to incorporate into the training set a lot of quasi-random or structured but abstract images as controls for training.