The point of TLS is to protect against MitM. The server refusing the non-TLS connection is not safe against MitM, a MitM can still claim to be the target server and accept the connection and thus (potentially) compromise the website/account/whatever. That is why the client has to refuse even trying to connect without TLS.