Yes, the author is clear in this regard. But I've seen people interpreting this paper as "training deep networks is chaotic", and I don't think that's the case. I interpret it more as "if you're not careful with your learning rate your training will be chaotic"

Yes. I immediately had visions of a misconfigured PID controller or similar chaotic emergence encountered in control theory. I was surprised and delighted though, that this type of chaos has so much fractal beauty.

Note that not all fractals are chaotic.

Inter dimensionality is the defining feature of fractals.

A multilayer ANN will have compression that may be fractal like. But in theory a feed forward network could be single layer but will a lot more neurons.

I have the feeling this result was due to representation but there are some features that look like riddled basins.

Riddled basins do arise in SNN or spikey neutral networks that have continuous output bs the binary output of ANNs.

But as all PAC learning is just compression that may be the cause also.

I'll be digging into this on the weekend though.

With all do respect, why are you commenting here rather than replying to the person giving your comment context?