The article states internet was down for users at around 10pm (I expirenced it live). So they might have been on generators for two hours (speculation).
Re:DE-Cix – Power failure leads to Downtime in germany
by AdamJacobMuller on Monday April 09, 2018 @10:47PM
(#16798644)
> Shouldn't we decentralize more?
> The whole of DE-CIX isn't down.
Largely agreed with you, though as general advice I would say that redundancy by adding more exchanges to your network serves you better than adding diverse connections to a single exchange.
Re:DE-Cix – Power failure leads to Downtime in germany
by virtuallynathan on Tuesday April 10, 2018 @10:53AM
(#16801782)
Yep, that makes a lot of sense, especially since 10G/100G waves and such on the FLAP route are so inexpensive. Getting a remote peering port on AMS-IX or something is probably a good move.
Re:DE-Cix – Power failure leads to Downtime in germany
by mmarx on Monday April 09, 2018 @06:57PM
(#16797709)
> The article states internet was down for users at around 10pm (I expirenced it live). So they might have been on generators for two hours (speculation).
More likely, the timestamp in the picture is UTC, which corresponds to 22:00 CEST, cf. [0]
Re:DE-Cix – Power failure leads to Downtime in germany
by walrus01 on Tuesday April 10, 2018 @12:27AM
(#16798986)
Well, now we know that significant portion of that facility are 1+0 for power systems. In a true 1+1 setup with air gap between A and B side power systems this wouldn't have happened.
Re:DE-Cix – Power failure leads to Downtime in germany
by AdamJacobMuller on Monday April 09, 2018 @10:40PM
(#16798620)
We were hit by this.
In fact, we're hit by this again, down again at 22:09:19 EST for me and still down. Their website is broken too (https://www.de-cix.net/) and their email servers are offline which makes contacting them amusing.
Not much to be done, really, fortunately the impact for us is very trivial.
Re:DE-Cix – Power failure leads to Downtime in germany
by avh02 on Tuesday April 10, 2018 @04:59AM
(#16799999)
General home connectivity also suffered, at least a few people (and myself) had connectivity issues, if you search for @vodafone_de on twitter you'll see a lot of people complaining (though I assume it was not just vodafone with issues)
Keep in mind this is only one of the many DE-CIX datacenters, they exist in at least 8 datacenters in Frankfurt: https://twitter.com/goebelmeier/status/983453047543214081
Only those peers who are single-homed to Interxion FRA5 / DE-CIX6 will have experienced an outage.
I am concerned that this even happened in a place important like this.
pic related: http://img.pr0gramm.com/2018/04/09/9286baa4b2261184.jpg
The article states internet was down for users at around 10pm (I expirenced it live). So they might have been on generators for two hours (speculation).
Shouldn't we decentralize more?
> The whole of DE-CIX isn't down.
Largely agreed with you, though as general advice I would say that redundancy by adding more exchanges to your network serves you better than adding diverse connections to a single exchange.
More likely, the timestamp in the picture is UTC, which corresponds to 22:00 CEST, cf. [0]
[0] http://status.core-backbone.com/index.php?id=1972
In fact, we're hit by this again, down again at 22:09:19 EST for me and still down. Their website is broken too (https://www.de-cix.net/) and their email servers are offline which makes contacting them amusing.
Not much to be done, really, fortunately the impact for us is very trivial.