Transatlantic cable down. Are they busy installing a prism?

Where is the USS Jimmy Carter?

For everyone else who didn't catch the reference, apparently this Seawolf-class submarine can tap undersea cables.

http://www.nytimes.com/2005/02/20/politics/20submarine.html

http://cryptome.org/eyeball/mmp/jimmy-carter.htm

They just want to redirect some traffic through the US for a while.

Yeah, that is kinda the point of having transatlantic cables in the first place :)

So this is the famous "losing hearts and minds".

> It was a scheduled job that went a bit wrong, says Bo Djurberg, manager at Telia

from http://www.aftonbladet.se/nyheter/article18915221.ab (via google translate)

The text [now updated] says it was solved at around 23:00 CET, after an hours outage.

> "One of our upstreams (Telia) appears to have a trans-Atlantic cable cut. We're routing around in many EU locations now." @cloudflare

https://twitter.com/CloudFlare/status/468494276691640320

I can pretty much only reach sites which are routed through the netherlands for me (like Cloudflare proxied HN, or Google). Everything else doesn't connect at all or times out after downloading 10% of the HTML. I'm connecting from Estonia.

The Digital Ocean status page [1] claims that "Preliminary investigation indicates that Telias Transatlantic cable are down." which would make sense, because my ISP is Telia and I can't connect to most of the internet.

Edit: Well, this outage lasted a bit over an hour. Now at 20:55 UTC I can access everything once again.

[1] http://www.digitaloceanstatus.com/

Its amazing how well traffic gets rerouted now days

Hi guys, I'm very curious about the effects of this. What is the network like? Can you characterize it in terms of latency, dropped packets, bandwidth?

Say if you wanted to upload/download a megabyte of data to a datacenter in the US using TCP with retries, would it work eventually with high probability?

It was really interesting how I could still communicate to some of the servers via connections that went through Level3.

Yet once I got back in the server (behind that indirection), the server couldn't even ping google, and its DNS was totally down.

Would've done a traceroute for fun if OVH's mirrors were reachable.

Ah, thanks for the info.

Connections to our US broker failed at 10pm (dst) on different servers (based in Germany) and i had no idea why, I already wanted to blame our hosting provider.

Clearly a test for who can withstand network partition...

Germany: Dropbox unreachable, Tumblr very slow

mostly unreachable from berlin: rackspace.com, amazon.com

but I see the traceroutes failing after they make it to the US.

Weird, why would it fail after reaching the US?

It could be misleading. During the outage, I saw traceroutes like:

    1 someserver.in.europe 10ms 15ms 20ms
    2 someother.in.europe 50ms 60ms 50ms
    3 * * *
    4 * * *
    5 * * *
    6 * * *
    7 * * *
    8 * * *
    9 someserver.in.us * 500ms *
    10 * * *
    11 otherserver.in.us 900ms * *

So, it looks like for 10 and 11 hops we are losing a lot of traffic in the US, but that might as well be traffic that's lost in hop #3. The traceroute only tells you at some point between the start and hop #11 we lost 66% of the packets. It doesn't tell you where.

You can also get * on traceroutes (step 3-8) because the devices don't respond to ICMP. Network admins like to disable ICMP on their switches and routers so you can't trace the network topology. Your only definite packet loss is on 9 and 11 because you only got 1 of the 3 packets returned.

Russia starting a cyber-war?

Actually recent news show that this is US who has the most control over the internet. But this outage could be natual disaster :-)