With Continuous Delivery you should be able to roll back the deployment. If the last change is the only one you need to revert. Of course that stops working when you share the flag system across teams with separate milestones. Your guys are gonna want to flip toggles while my guys are thinking about flipping others.