you can set up a trigger port, that will open an SSH port when triggered properly. some routers come with this functionality, a third party opensource firmware usually does this or allows for a script to implement it.

so you "knock" on port 666 lets say with a majic packet of some sort this opens a secured SSH port, then drops it on command or idle time to live expires