> The next trick I think of implementing is port knocking.

If you're at that point, I would suggest putting it behind wireguard.