We use fwknop (time-based, single-packet, single ip, port knocking with gpg keys) + monkeysphere (ssh keys stored in gpg) + Google Authenticator on FreeBSD with PF enabled in addition to Security Groups on AWS.