They can't do a man in the middle attack because the transaction is already signed with your private key and they don't have that.