> If one rejects the ERH, one could argue that our universe is somehow made of stuff perfectly described by a mathematical structure, but which also has other properties that are not described by it, and cannot be described in an abstract baggage-free way. This viewpoint [...] would make Karl Popper turn in his grave, since those additional bells and whistles that make the universe non-mathematical by definition have no observable effects whatsoever.

I don't think that follows. It could be that the universe is asymptotically mathematical, in the sense that any mathematical structure falls short of perfectly describing the universe, but there is always a more sophisticated mathematical structure that is a closer approximation. The problem of course is that a mathematical description is made of a finite number of symbols. It could be that the external reality hypothesis holds, but the universe can only be described in a baggage-free way with an infinite number of symbols.