If you have the luxury to do that, but if you run a VM, or a single box in colocation, or rent a physical server, that may not be an option. My point is introducing an IP whitelist inside the server doesn't require to change the physical set up, nor any current process relying on the server being directly accessible. It's a low cost quick win.