You aren't infallible. While it's easy enough to secure a minimal server with only SSH access, every time you add another service, provide access to a new user, or change a configuration, there's a chance you'll miss something or make a dumb mistake. Running Fail2Ban is a very cheap and easy method to ensure that, even if you do fuck up somewhere, there's another layer of security that should prevent malicious access in most cases (nothing will save you if you set the pass to admin/admin, e.g.).