Ahhh, yes parts of that sound familiar. So, the reason we can (at least pretty much) ~~rule out~~ a 3-sphere [EDIT: I didn’t read carefully, and missed the “it could be a really really big 3-sphere so that the curvature is close enough to zero” part], is because you can’t have a flat sphere,

But you can have a flat torus (or some other shapes that “wrap around”), but we have different reasons to disbelieve those shapes (the “looks the same in any direction” and “looks the same in every position” expectations).

Cool, thanks!