It's not a problem; the rules only apply to eth0, but I connect over tun0. And, it's time-limited to 60 seconds either way. And, I have web console access.

So this is not something I worry about. ssh password crackers and script kiddies taking down my website are a little more worrisome.