That is indeed where the analogy starts to crumble. The reason is that space itself is curved. It is curved by the matter inside it. Still the property holds that if you travelled far enough fast enough you’d come back to where you started.