Re:Google Cloud Networking Incident Postmortem
by vorpalhex on Friday June 07, 2019 @10:00AM
(#20124543)
Some equipment will auto-revert to a last known good configuration if you don't approve new changes within a window... though high CPU could lock that process up..
[ Parent
| on HN ]
Re:Google Cloud Networking Incident Postmortem
by riskrisk on Saturday June 08, 2019 @02:14AM
(#20130932)
in this case the old configuration was lost, it took an hour to rebuild because tooling normally used to rebuild it for testing was unavailable and building had to be done locally, using a single machine someone ssh-ed into, and that just takes a while. Luckily, a person was around who knew how to do the rebuild without fancy tooling.