If you have more than one backend you can also reconfigure caddy on the fly to only serve from active ones while each one is being updated