Or you do what Matrix does, and given every device its own keypair, and let users track whether they are talking to a trusted decice or not.