Sure, and that'd be fine for a special-purpose app that you use for yourself, but imagine being a regular user and installing this app, and the author wants you to install something on both the client and server side, and to run ssh in this unusual way. And you have to ensure that you have the proper permissions for raw access to the keyboard on the client side.

If it's something that truly requires that sort of experience to work properly, ok, but if it's just a sort of "I want a fancy text UI that requires key-up events", this process is a bit much to ask of your users.