News for hackers. Stuff that matters.

Amazon EC2 EU-West is down

Nothing yet on the AWS service status but Twitter is going a bit crazy.

UPDATE 19.15 BST: now being reported as network issues by Amazon

UPDATE 19.24 BST: Can now see my ec2 instances again from the management console but can't ssh into most of them.

UPDATE 19.27 BST: From Amazon: EC2 APIs in the EU-WEST-1 region are currently impaired. We are working to restore full service. We are also investigating instance connectivity that we believe to be limited to a single Availability Zone.

UPDATE 19.34 BST: RDS and Route 53 also affected.

UPDATE 20.10 BST: From Amazon: The issues with the affected Availability Zone are the result of a power failure in that zone. We are currently recovering power and anticipate that instances in the effected available zone will start to recover within the next 30-60 minutes.

Well, that sucks.

( 31 comments | 109 points | on HN )
Re:Amazon EC2 EU-West is down by jread on Sunday August 07, 2011 @03:29PM (#2857500)
I run a site that monitors over 100 different cloud services. Our monitoring node in EC2 EU West went down at 10:48:51 PDT. Our monitored application with PHP PaaS cloudControl, also hosted in EC2 EU West, went down at the same time. For the past year, we've gotten 99.978% availability in this EC2 region: http://cloudharmony.com/status
Re:Amazon EC2 EU-West is down by jimmyjim on Sunday August 07, 2011 @03:56PM (#2857571)
Power outage?

Interesting -- Linode also went down today because of power outage: http://status.linode.com/2011/08/outage-in-fremont-facility....

Re:Amazon EC2 EU-West is down by whakojacko on Sunday August 07, 2011 @05:18PM (#2857761)
This is what, the 3rd? 4th? power outage in that facility in the last year? I realize that it's not directly Linode's fault, since this is Hurricane Electric's facility that they lease from, but its still pretty frustrating. I have a personal VPS there which I use to do an hourly data grab, so I missed a few this morning. This is just a personal pet project so no huge deal, but if I was running some actual production-level stuff there I would be very wary of using that facility.
Re:Amazon EC2 EU-West is down by bradly on Sunday August 07, 2011 @04:13PM (#2857607)
Is there any chance these power failures have something to do with the solar storms we are experiencing right now?
Re:Amazon EC2 EU-West is down by techiferous on Sunday August 07, 2011 @04:20PM (#2857626)
They have subsided. Note the K-index on this page: http://spaceweather.com
Re:Amazon EC2 EU-West is down by getsat on Sunday August 07, 2011 @06:11PM (#2857908)
Linode doesn't notify their customers affected by outages, or am I missing something?
Re:Amazon EC2 EU-West is down by hrasm on Sunday August 07, 2011 @09:25PM (#2858365)
They don't. They do have an RSS feed that you can monitor with your favourite feed aggregator: http://status.linode.com/rss.xml
Re:Amazon EC2 EU-West is down by getsat on Monday August 08, 2011 @06:42AM (#2859533)
Thanks for the link.
Re:Amazon EC2 EU-West is down by Macha on Sunday August 07, 2011 @06:50PM (#2858033)
I've been notified for what amounted to two minutes of downtime before (my VPS is in the London data centre). Are you in the Fremont data centre? That is where this outage was.
Re:Amazon EC2 EU-West is down by getsat on Monday August 08, 2011 @06:42AM (#2859531)
Yes, I have a machine in Fremont. How were you notified? Email?
Re:Amazon EC2 EU-West is down by Macha on Monday August 08, 2011 @09:48AM (#2860046)
Yes. An email about unexpected downtime, then another two minutes later saying the downtime was over.
Re:Amazon EC2 EU-West is down by getsat on Monday August 08, 2011 @05:08PM (#2861590)
Hmm, I don't see any way in my Linode account settings to disable these emails, and I never received any! That is disconcerting. :(
Re:Amazon EC2 EU-West is down by puredemo on Monday August 08, 2011 @02:30AM (#2859057)
So that's what interrupted my screen session.
Re:Amazon EC2 EU-West is down by chaz on Sunday August 07, 2011 @05:19PM (#2857767)
A must-follow Twitter feed for anyone using Amazon AWS is @ylastic: http://twitter.com/#!/ylastic

At the very least, even if you don't use Twitter, it's a good resource to check for up-to-the-minute info if you're seeing an issue and want to verify.

Re:Amazon EC2 EU-West is down by bambax on Sunday August 07, 2011 @04:08PM (#2857596)
THANK YOU HN!!! I couldn't connect to my machine and couldn't understand what was going on... good thing I turn to HN as soon as I get online...

You'd think AWS could afford to send emails to keep us customers up to date with the problem?

Re:Amazon EC2 EU-West is down by rwolf on Sunday August 07, 2011 @04:22PM (#2857631)
If they sent you an email every time there was downtime, then you would know every time there was downtime. Any time your servers are not available but you don't notice == win.
Re:Amazon EC2 EU-West is down by bambax on Sunday August 07, 2011 @04:29PM (#2857647)
> Any time your servers are not available but you don't notice

Can't happen. My servers are monitored by external services that (to the best of my knowledge) don't use AWS.

It's best to play the open card in any case.

Re:Amazon EC2 EU-West is down by spartango on Sunday August 07, 2011 @05:15PM (#2857751)
Emails will be sent out when the problem is more fully handled, as usual. HN is a decent place to get some info, but you should consider posting to the AWS developer forum as well, b/c there are Amazon people constantly checking on things and answering questions:

https://forums.aws.amazon.com/forum.jspa?forumID=30

Edit: I should add, they can look up your specific instances to get the exact problem.

Re:Amazon EC2 EU-West is down by 1SockChuck on Sunday August 07, 2011 @09:01PM (#2858313)
Looks like a lightning strike in Dublin knocked out power at data hubs for Amazon and Microsoft: http://news.ycombinator.com/item?id=2858239
Re:Amazon EC2 EU-West is down by stephank on Sunday August 07, 2011 @03:07PM (#2857444)
The specific availability zone they're talking about is eu-west-1a. Servers we have there are reported as 'running', but are unresponsive.

Other zones seem fine, but starting new servers doesn't seem to be working.

Re:Amazon EC2 EU-West is down by amock on Sunday August 07, 2011 @03:12PM (#2857454)
Availability zones are different for different accounts, so for me it's eu-west-1b. You can log into the console and under the EC2 tab it will show you the status of each of your availability zones.
Re:Amazon EC2 EU-West is down by stephank on Sunday August 07, 2011 @03:14PM (#2857460)
Ah, good to know, thanks for the pointer.

Another update on the status page, btw. It appears to be a power failure.

Re:Amazon EC2 EU-West is down by sdepablos on Sunday August 07, 2011 @03:22PM (#2857481)
I've tried to create a machine in another zone with no success. I suppose there's a lot of people doing the same :(
Re:Amazon EC2 EU-West is down by calpaterson on Sunday August 07, 2011 @05:58PM (#2857860)
If anyone is interested, ServerDensity thinks that our machine went down between 6:37 and 6:42 BST.
Re:Amazon EC2 EU-West is down by rmc on Monday August 08, 2011 @07:35AM (#2859650)
As an AWS customer affected by this, that's the time our servers went dark.
Re:Amazon EC2 EU-West is down by dabeeeenster on Sunday August 07, 2011 @02:17PM (#2857306)
I can SSH to some of my newer instances but not the older ones.
Re:Amazon EC2 EU-West is down by sdepablos on Sunday August 07, 2011 @02:53PM (#2857404)
I see all my servers on the management console, but I'm unable to ssh any of them, all in the same availability zone, eu-west-1b
Re:Amazon EC2 EU-West is down by zizzler on Monday August 08, 2011 @11:00AM (#2860259)
Has anyone gotten any timeframe when the rest of the RDS is going to start responding? Not much info on the status page of AWS.
Re:Amazon EC2 EU-West is down by pimeys on Sunday August 07, 2011 @02:36PM (#2857361)
I can see my server from the management console but I can't ssh to it. Damn, my IRC is down :(
Re:Amazon EC2 EU-West is down by dialtone on Sunday August 07, 2011 @02:59PM (#2857420)
CloudFormation is also affected now.
Re:Amazon EC2 EU-West is down by sipefree on Sunday August 07, 2011 @02:36PM (#2857364)
Clear skies over Dublin, not a cloud in sight. ;)