Sure, you can probably come up with some non-HTTPS scheme that can address this, but I don't see any site actually doing this, so you're back to the unrealistic scenario.
If I were trying to phish someone, I wouldn't attack the public key crypto part, so how domains come into play during authentication doesn't matter. I'd just grab the "unencrypted" session token at the end of the exchange.
Even if you somehow protected the session token (sounds dubious), there's still plenty a phisher could do, since it has full MITM capability.
There's also the issue of how many sites actually use it, as well as how it handles the loss of or inability to access private keys etc. I generally see stuff like 'recovery keys' being a solution, but now you're just back to a password, just with extra steps.
Except that phishing doesn't require the private key - it just needs to echo back the generated token. And even if that isn't possible, what stops it obtaining the session token that's sent back?