Wire is open source. It'd be a pain to self-host, but it's doable. And at the very least, you can verify that your client is indeed using E2E encryption and therefore trusting the server isn't necessary.

https://github.com/wireapp