The man in the middle can do the key exchange with you. (You seem to be assuming that you start with a connection that you know to be reaching the desired endpoint and that you can start the key exchange there. You can't. The MITM intercepts the initial connection and does the key exchange with you and then turns around and initiates a secure connection with the eventual endpoint. The endpoint sees valid, encrypted traffic and you see valid, encrypted traffic, but the MITM attacker has the full plaintext communication.)

That's why you need the identity portion to be tied to the key exchange.