If you take the number like and apply f(x) = 2x, it was infinite before and after.
The number line is not finite, even though every value on it is. The universe may be like the number line.
> and given that it started out being very small in a big bang, it would imply the former
The universe started out denser, which is not quite the same as "small" when talking about infinities. The currently visible universe gets both things at the same time, but only because the visible bit is finite.
Apply f(x) = x * 1e-60 to the number line, and things which were previously spaced out every integer, are now much more densely packed, but the number line as a whole is still the same size — infinite.
f(x) = 1/(1-x)
So if x is finite, f(x) can still be infinite.
Anyway, we don't actually know that, since we do not have a time machine. The theory that the universe used to be much smaller is something we infer based on the snapshot we see today, just like we infer rotation based on the snapshot we see today. Either one could yet be proven wrong.