I looked into it a couple of times, but most clients lack E2E encryption support, and the oficial ones are Electron monsters. We mostly work over SSH on remote workstations, hence the CLI/TUI requirement.

Altough, it seems there is a plugin for WeeChat with E2E encryption[1]. Gotta take a look.

[1] https://github.com/poljar/weechat-matrix