If your utilities are written as self-contained exported Bash functions, you could concatenate them all in a single file (say, `utils.sh`), with a single line at the end:

    exec /bin/bash

Then you could do something like:

    ssh hostname@address -tt bash -s < /path/to/utils.sh

And have all your utilities available on the remote machine :)

Does this work for aliases?

By "personal computing" I guess I do mean machines that I have full root on, so usually either a laptop/desktop or a local headless server.

Out of curiousity, these servers that you're dealing with, do you have to use them for long/focused work like writing or software development, or are these more like sysadmin interactions? Have you noticed any repeated commands or sequences you find yourself running over and over? Is there any opportunity for you to write local scripts that then SSH into the targets and run those commands on your behalf? Or locally use sshfs and manipulate files like they were local.