If the hosts are under your control, and never connect to untrusted hosts, then you are ok. The user authentication is encrypted, so the signatures are not visible to a man in the middle.