Machines that can't run Matrix is a pretty small subset. There's a Matrix plugin for Weechat, so you should be able to connect from anything with roughly the power of a Raspberry Pi. Probably slightly less than that; but yeah, you're probably not going to have a good time trying to use an ESP32 as a client. I don't see that as a huge drawback.