It's an assumption that someone is happy with mathematical structures being infinite.

There are mathematicians unhappy with the notion even while being well versed in the Cantor (and other) arguments.

You can wander down that rabbit hole via (say) [1] as a starting point.

[1] https://en.wikipedia.org/wiki/Constructivism_(philosophy_of_...

Because universe is a physical structure, not mathematical, e.g. you can't define things into existence.

Is it? Says who? This is an untirely unfalsifiable assumption. I personally don't share it. "It from bit"[0], I say, mathematics is fundamental, physical reality is just our perception of the underlying math.

[0] Title of a paper by John Wheeler. Can't find it right now, but if you google "it from bit" you'll find a lot of commentary on it.

The concept of fundamentality doesn't exist in mathematics, and it's by design. What is fundamental, Tailor series or Fourier transform? Mathematics doesn't care, it simply has no such concept.

Found commentary on "it from bit". It looks like an attempt to resurrect antirealism (participatory universe). The reason why antirealism was buried is because there's too much realism in quantum physics, so it won't fly.

This is exactly the position I'm trying to argue against.

What does it mean for something to be a physical structure? That you perceive it to be physical? But aren't your perceptions part of the universe? So how would you know if it actually "exists" anywhere?

Physical structure works by its own rules, mathematical structures work by our rules. For example, in mathematics you can complete an infinite process, because there's no time in mathematics, it can just be assumed out of existence. It can't be done with physics, because physics isn't affected by our assumptions.

Time and physics are just things you experience. They're parts of the universe, they're within the universe. The mathematical universe itself can quite easily be infinite in both space and time, you just can't experience all of it.

Physical systems have finite information storage capacity. Our infinities are descriptions of recursive processes (either iteration or subdivision). The quantization we've observed in the universe is a strike against infinities of subdivision, and it's literally impossible to observe a macro-infinity, so from a scientific perspective physical infinity is unsupported and seems unlikely.

Space might appear "infinite" if you set up a word problem where you fire two photons off in opposite directions and track the distance between them over time, but without an observer the only place information exists is in those photons, and space is meaningless. Infinities of time are likewise artificial constructs of measurement, there is only the current moment.