For example, the Euler equations model compressible flow with discontinuities (shocks in the flow field variables) and rarefaction waves. These theories are accepted and used routinely.

Great. A useful approximation of what really happens in the fluid. But I'm sure there are no shocks and rarefactions in physicists' neurons while they are thinking about it.

Switching into a less facetious mode...

Do you understand that in context of this dialogue it's not enough to show some examples of discontinuous or otherwise unrepresentable by NNs functions? You need at least to give a hint why such functions cannot be avoided while approximating functionality of the human brain.

Many things are possible, but I'm not going to keep my mind open to a possibility of a teal Russell's teapot before I get a hint at its existence, so to speak.