I used to have an iptables config that just drops everything by default on the SSH port and run a DNS server that when queried a magic string would allow my IP to connect to SSH. It did help that the DNS server was actually used to manage a domain and was seeing traffic so you couldn't isolate my magic queries so easily.