Re:AWS power failure in US-EAST-1 region killed some hardwar...
by lamontcg on Friday December 24, 2021 @01:00PM
(#29675909)
I worked at Amazon from 2001-2006 and there were a couple of datacenter-wide power outages during my time and there was always some machines that never came back. It is just a numbers game. When you have thousands of servers affected some small fraction just don't survive the forced power cycle.
Re:AWS power failure in US-EAST-1 region killed some hardwar...
by jbergens on Thursday December 23, 2021 @05:15PM
(#29667125)
A very long time ago I worked for a medium sized company and there was a power outage. The servers were in a room in our building but they had ups power. The problem was that the ups's were broken/malfunctioning and the servers needed to do a disk check after an unplanned reboot. It took hours before we could work again even though most drives survived.
This was also before git so we could not reach the central version control system and could not commit any code. I think all code and workspaces were on servers that was not up yet.
I remember IBM was renting private planes to get us replacement hardware from around the country (we were a very large customer).