News for hackers. Stuff that matters.
Re:Google OAuth Login Issues by aaronharnly on Wednesday April 17, 2019 @07:47PM (#19687431)
Our incident management includes chatops automation that, among other things

- generates a per-incident Google Doc

- opens a per-incident Google Hangout

- pages appropriate people via PagerDuty

During this incident (which affected our customers who use Google to log in), people had difficulty accessing all three (our PagerDuty uses Google Oauth).

This was a good reminder to have alternatives for all crucial incident management services. (We have backup plans in case Slack or our automation scripts are down, but didn’t have backups for Google...)

Re:Google OAuth Login Issues by shhehebehdh on Wednesday April 17, 2019 @07:49PM (#19687445)
Good advice. Within Google, they have backups that depend on no production infrastructure. Something as simple as an IRC server that is always running on prem can be invaluable.
Re:Google OAuth Login Issues by ViViDboarder on Wednesday April 17, 2019 @08:21PM (#19687627)
Can’t emphasize this enough! Outages will eventually happen, you need to plan for them.
Re:Google OAuth Login Issues by lathiat on Thursday April 18, 2019 @09:25AM (#19690792)
One of canonical’s employees was about to do a fairly important presentation (a few years ago) when our google oath went down. Could no longer load the slides from google docs and the had to make up new ones on the spot.