Re:Nov 16 GCP Load Balancing Incident Report
by lrem on Tuesday November 23, 2021 @01:58AM
(#29315263)
As a Googler privy to the internal postmortem: as stated in the public postmortem, all traffic was unaffected within 33 minutes of the problem appearing. The bug was very on/off: at 09:35PT a corrupted configuration stopped ~immediately (usually double digit seconds of propagation delay) all traffic. At 10:08PT it was verified that the whole service is running the configuration from before the corruption.
The >1h duration was for inability to change your load balancing configuration.
Re:Nov 16 GCP Load Balancing Incident Report
by roytries on Tuesday November 23, 2021 @03:30AM
(#29315761)
Maybe you're thinking of this incident? https://status.cloud.google.com/incidents/1xkAB1KmLrh5g3v9ZE.... It was a few days earlier and took almost 2 hours.
Re:Nov 16 GCP Load Balancing Incident Report
by cajones314 on Tuesday November 23, 2021 @12:45AM
(#29314952)
We received errors at least 45 minutes before their stated time. :-/
Re:Nov 16 GCP Load Balancing Incident Report
by lrem on Tuesday November 23, 2021 @02:22AM
(#29315385)
Then you have been hit by some other issue.
My company wasn’t effected, so I wasn’t paying close attention to it. I was surprised to read it was only ~90 min that services were unreachable.
Anyone else have stabilizing ancedata?