I don't find it super interesting. The orientation distinction is clearly in the training data, and making the algorithm completely rotation invariant would likely a) be more difficult and b) result in worse classification compared to us humans who very much use orientation as a cue having been evolved in an environment with distinct "up" and "down" directions.

As the AI has likely not seen anything remotely similar during training, it is quite interesting that it is detecting the animals. As the picture was set up to confuse humans, does it somewhat show, that the representation that the AI learned is similar to the one humans have?

The AI has seen ducks and rabbits. It has build a good enough internal model of "duckness" and "rabbitness" that even more abstract renderings of ducks and rabbits activate the relevant parts of the net. I mean, that's exactly what we want them to do! Figure out the aspects of the input data salient to the classification task and ignore the irrelevant parts.

That said, I the response here is likely filtered and normalized to only include "duck" and "rabbit" classifications; after all the bird looks much more like a seagull than a duck.