There are attacks where all pixels are just slightly changed and the classification is completely wrong. Think of changing all pixels by just the least significant bit. Still these changes are invisible to (my) human eye. Basically, each distance function has atttacks with a very small distance.