News for hackers. Stuff that matters.
Re:The dangers of resetting everything at once by grecy on Monday May 04, 2015 @12:52PM (#9487353)
I work at a telco with a lot of aging systems and servers.

A couple of years ago we had a total power failure of our major data center and central office, including the backup gen so literally everything failed and an entire province of Canada (and most of two more) were without any telco, data, wireless.

It was eye opening and extremely scary to see what happens when everything is turned off at once and you need to bring stuff back up. Some of the equipment had never been turned off, and certainly not all at once.

During the next weeks we encountered many, many catch-22 scenarios like "Before we turn on System A, it needs system B to be online. B needs C, C needs A." Oh.

For reference of how severe this is, paramedics were posting up at major intersections in towns and communities in case people needed help - and the radio stations in half of Canada were telling people what to do. When even 911 doesn't work you have problems.

Re:The dangers of resetting everything at once by serve_yay on Monday May 04, 2015 @03:25PM (#9488444)
This is how "if it ain't broke, don't fix it" calcifies. People point to old systems like this (when they are working, that is) all the time as proof that developers are too obsessed with shiny new things. And I'm not saying that isn't true, but there is such a thing as being too conservative.
Re:The dangers of resetting everything at once by grecy on Monday May 04, 2015 @04:07PM (#9488773)
> there is such a thing as being too conservative

In my experience, a telco is the definition of "too conservative".

Re:The dangers of resetting everything at once by OldSchoolJohnny on Monday May 04, 2015 @01:25PM (#9487581)
As a Canadian I would love to see a citation to this because it would have been pretty big news here and I recall nothing about it.
Re:The dangers of resetting everything at once by grecy on Monday May 04, 2015 @01:54PM (#9487792)
http://www.ctvnews.ca/canada/power-outage-in-yukon-knocks-ou...

It took out everything in the Yukon, and tons of stuff in NWT, Nunavut and Northern BC and Alberta.

Re:The dangers of resetting everything at once by digi_owl on Monday May 04, 2015 @04:40PM (#9489030)
I guess the catch-22 came into being because A was at some point bootstrapped from some other system that was since replaced by B or C.