To me, mosh is "careful and impressively pedantically correct UTF-8 terminal emulator" + "careful and impressively pedantically correct state synchronization protocol". :)
UI is just plain ol' things for user to interact with computer.
If it were like ssh the authors would have had to then handle security/authentication and all that jazz perfectly. It was written to "stand on the shoulders" of ssh, except handle terminal emulation and UTF-8 correctly. By using ssh, you don't have to trust "some new thing" as long as you trust ssh.
It's not even a shell protocol if you think about it!
Mosh bidirectionally synchronizes the state between the terminal window on the client and the virtual terminal on the server. It runs at a frame rate, which results in not filling intermediate network queues, which is where the low latency comes from. :)
Nor is SSH. Feel free to take a read over the RFCs. There's next to no 'sh' in SSH as a protocol. What mosh adds to SSH is the ability to securely resume a session. SSH is fundamentally protocol for setting up a secure connection and then ping-pong messages back and forth.
tmux runs locally. It's not a network daemon. The complexion of a daemon and protocol meant to operate over a network is very, very different from one intended to run over Unix domain sockets or on the loopback interface.
without needing to run mosh on the server