You're right but actual good server providers take it up a notch and allow you to upload the public key through their admin panel during server set up and the password you get only works locally for sudo etc. There's no reason why a fresh Linux setup should ever offer a publicly available SSH server with password based auth out of the box.