> someone will scan it and try to attack it. Maybe not if it's a random game server, but any popular service will get under attack.

That's fine, it's only people knocking on a closed door. You cannot host things such as email or HTTP without open ports, your service needs to be publicly accessible by definition.