Using a short ban time keeps the load of running fail2ban to a minimum. I at one point was banning the kiddies/bots for a week and 40% of the server load was chewed up just running fail2ban.
The solution I came up with to this problem is to use a custom ban action that adds an IP to the blocked ipset list with a given TTL, but set fail2ban to think that the IP has to be only banned for 5 seconds or so.
The result is that ipset purges the IP from the list after the TTL expires, but fail2bans working set is a 5 second rolling windows of blocked IPs.
This works great and it's being used in production for a couple of years.
> so that my logs don’t fill with a zillion pointless attempts
What about moving SSH to a different port?You don't really need fail2ban at all. Just don't log failed password attempts. Key access only, no one is going to get in.
Of course there are ways to deal with it, but that's just extra work.