No, it's essentially the same thing as having a password-reset email sent to one, which is already the case for just about any account anyway.

> You're putting the responsibility of security on the email provider, which you can't control.

Your email provider is already capable of visiting a site, claiming to be you & claiming to have forgotten your password, then intercepting the password-reset email and resetting your password (about the only sites which require more than this are banks).

You can extend any flow which is secure against an evil email provider resetting a password to be equally secure against an evil email provider logging in. Obviously you could add an SMS code, which would then require subverting your mobile provider as well as your email; ditto a landline call.