Check out the 34C3 talk on adversarial AI. They found that the percentages of fooling are still high if one gets the adversary model completely wrong when designing the attack generating network, so it seems surprisingly stable.

I think that working around these type of fooling is easy but not really worthwhile for now. After all, adversarial models are designed to improve the performances of the models.

Also in the article, they test a detector that has to identify a single object in an image that contains two: place an actual toaster next to the banana and call it fooled.

The point there may have been that the sticker over-powered the banana. The article does leave me with more questions than answers.

Ask them, I may have some answers.

Exactly!

More than that, you can fool models that work completely differently (like decision trees, SVM and kNN) with false data made for the other model, which shows some kind of underlying similarity in there that we don't know yet.

Or maybe just similarity of the training sets?

Do you know how to fool the detector in the article even if it were perfect? Put a sticker with a picture of a toaster that is closer to the toasters in its training dataset than the banana is, to the bananas in its training dataset.

Humans would consider this a non-interesting exploit.

This detector has to choose a label for an image with two labels. Use something like YOLO2 on this, and the detector will recognize a banana AND a toaster.

Now we do know that compared to humans, these detectors over-react to textures over structure. If you look at the sticker, you can see how it kinda looks like a toaster: the big red blob looks a bit like a toaster button. A generic shape is there, the thing over the button looks like both the control to lower the toasts and the slit where the toasts are.

These classifiers will get better at recognizing structure, especially if we train them against this kind of things.