News for hackers. Stuff that matters.
Re:Amazon Web Services are down by jedberg on Thursday April 21, 2011 @06:00AM (#2469970)
Yes, they are. :(
Re:Amazon Web Services are down by athesyn on Thursday April 21, 2011 @06:11AM (#2470016)
My four-day weekend is already off to a bad start(UK here).
Re:Amazon Web Services are down by G_Morgan on Thursday April 21, 2011 @07:59AM (#2470281)
Mine is worse. I booked Tues-Thurs off. I only have internet in work at the moment. I'm going to miss reddit now and be without internet until I return to work on the 3rd of May. Stupid Sky and their stupid take forever switch overs.
Re:Amazon Web Services are down by taitems on Thursday April 21, 2011 @06:20AM (#2470047)
It could be worse/better, you could have an Australian 5-day weekend.
Re:Amazon Web Services are down by buyx on Thursday April 21, 2011 @06:50AM (#2470117)
Or a South African 11 day weekend. Easter, and then public holidays on 27 April and 1/2 May are combining this year to provide a massive holiday opportunity.
Re:Amazon Web Services are down by mike-cardwell on Thursday April 21, 2011 @06:54AM (#2470122)
UK Universities give staff the Tuesday too, giving them a five day weekend. I took two days holiday next week and because of Easter and the Royal Wedding that gives me 11 days off straight.
Re:Amazon Web Services are down by endgame on Thursday April 21, 2011 @07:00AM (#2470136)
To clarify, this Monday is ANZAC Day, a commemorative holiday for troops who fought for Australia. Because that's also Easter Monday, the ANZAC Day holiday is moved to Tuesday, despite commemorative services being held on Monday.
Re:Amazon Web Services are down by kristianp on Thursday April 21, 2011 @08:15AM (#2470327)
Actually the official line is that Easter Monday got moved to Tuesday.
Re:Amazon Web Services are down by endgame on Thursday April 21, 2011 @09:09AM (#2470460)
Interesting. Thanks. What did the Catholic Church have to say about that? Is it that Easter Monday is still on Monday, but the holiday is on Tuesday?
Re:Amazon Web Services are down by spydez on Thursday April 21, 2011 @02:21PM (#2471674)
Easter is the holiday, and it's on Sunday. I'm pretty sure the Pope doesn't care much what people do the day after Easter (or the day after that).
Re:Amazon Web Services are down by endgame on Thursday April 21, 2011 @05:41PM (#2472508)
Ah, thanks. I misread the wikipage, and didn't realise.
Re:Amazon Web Services are down by mikecane on Thursday April 21, 2011 @10:57AM (#2470829)
Is it a 4-day holiday in Norway too?! Opera Support Forums have been sketchy for hours and if they won't come back until Tuesday, I'm SOL with my Opera problems.
Re:Amazon Web Services are down by nso on Friday April 22, 2011 @04:22AM (#2473551)
5-day
Re:Amazon Web Services are down by jedberg on Thursday April 21, 2011 @06:15AM (#2470027)
Why do you get a 4 day weekend?
Re:Amazon Web Services are down by bhrgunatha on Thursday April 21, 2011 @06:38AM (#2470092)
Whichever date Easter Sunday falls on, the Friday before it (Good Friday) and Monday after (Easter Monday) are both public holidays.
Re:Amazon Web Services are down by griosa on Thursday April 21, 2011 @06:50AM (#2470115)
Getting the 26,27,28th off from work is just genius for a long break!
Re:Amazon Web Services are down by sushi on Thursday April 21, 2011 @06:17AM (#2470034)
Easter

Royal Wedding

May Day

Re:Amazon Web Services are down by ahlatimer on Thursday April 21, 2011 @06:17AM (#2470033)
Easter, probably.
Re:Amazon Web Services are down by griosa on Thursday April 21, 2011 @06:05AM (#2469991)
no reddit at work today!
Re:Amazon Web Services are down by jedberg on Thursday April 21, 2011 @06:07AM (#2469997)
Believe me, the last thing I want is to be up at 3am working on this. I'd much rather be sleeping and letting you not work.
Re:Amazon Web Services are down by chrislomax on Thursday April 21, 2011 @06:22AM (#2470051)
Are you part of the team resolving it?
Re:Amazon Web Services are down by masklinn on Thursday April 21, 2011 @06:53AM (#2470120)
"Resolving" probably isn't the right word seeing as this is a purely amazon issue, not much they can do.

I'm guessing jedberg is mostly banging his head in walls and seriously looking at alternative hosting solutions right now.

Re:Amazon Web Services are down by jedberg on Thursday April 21, 2011 @07:36AM (#2470223)
Actaully, I'm on the couch in front of the fireplace, watching old SNL, waiting patiently for Amazon to fix their shit, and figuring out how we can not use EBS anymore.
Re:Amazon Web Services are down by Andrenid on Thursday April 21, 2011 @09:07AM (#2470453)
I'm pretty sure redditors would be more than happy to deal with a day or so extra downtime as you guys switched to a better platform. Just leave a simple page up saying "Dumping Amazon, brb"... doubt you'd get many complaints.
Re:Amazon Web Services are down by G_Morgan on Thursday April 21, 2011 @07:57AM (#2470280)
TBH Amazon is so bad at this point that turning off Reddit is as good as trying to keep it running. Of course then you need to deal with the increased suicide rate.
Re:Amazon Web Services are down by dwc on Thursday April 21, 2011 @10:30AM (#2470723)
Right on.

Periodically, latest a couple of days ago, there's a post / discussion about whether outsourcing core functionality is the right thing to do. There are valid points on both sides of the issue.

For my part, if I'm going to be up in the middle of the night I'd rather be up working on fixing something rather than up fretting and checking status. But either way things get fixed. The real difference comes in the following days and weeks. When core stuff is in the cloud then you can try to get assurances and such, fwiw. When core stuff is in-house then you spend time, energy and money making sure you can sleep at night.

Re:Amazon Web Services are down by alphamerik on Thursday April 21, 2011 @08:14AM (#2470326)
A couple of years ago you had expressed interest in making a port to App Engine, any interest in doing that still? Want any help? ;)
Re:Amazon Web Services are down by jedberg on Thursday April 21, 2011 @10:11AM (#2470660)
I think it would take a lot more time than we have to make that work. Our code is open source if you want to give a proof of concept a go. ;)
Re:Amazon Web Services are down by ultrasaurus on Thursday April 21, 2011 @12:03PM (#2471118)
On the off chance a port to app engine coalesces around this comment, count me in :)
Re:Amazon Web Services are down by masklinn on Thursday April 21, 2011 @07:44AM (#2470243)
So I was half right? Awesome!
Re:Amazon Web Services are down by chrislomax on Thursday April 21, 2011 @07:06AM (#2470148)
I was looking at EC2 until this!

I thought you could cluster your instances across many regions and replicate blah blah blah and change your elastic ip addresses in instances like this?

Is this a case that it's not being utilised or does that system not work?

I appreciate you are busy right now so I'm not expecting a reply any time soon.

Re:Amazon Web Services are down by jedberg on Thursday April 21, 2011 @07:34AM (#2470220)
That is the theory, but all of our data is currently locked in the inaccessible EBS system.

I'd still say Amazon is a great place for your startup, just don't use EBS.

Re:Amazon Web Services are down by chrislomax on Thursday April 21, 2011 @07:54AM (#2470269)
I thought you could snapshot drives across regions and bring those EBS drives up under new instances in new regions?

I have not used it all in detail yet so I don't know the practicality of this method.

I think I will stick to my co-location costings I am doing for the time being. There is only one person to rely on when it all goes wrong then!

Good luck getting it sorted, i know I wouldn't appreciate being up at 3am sorting it though

Re:Amazon Web Services are down by jedberg on Thursday April 21, 2011 @09:42AM (#2470559)
> I thought you could snapshot drives across regions and bring those EBS drives up under new instances in new regions?

In theory, yes. In practice, those snapshots hurt the volume so much that it is impossible to take one in production.

Re:Amazon Web Services are down by chrislomax on Thursday April 21, 2011 @10:08AM (#2470651)
Interesting, your insight has given me a lot to think about.

Do you guys blog this anywhere?

Re:Amazon Web Services are down by jedberg on Thursday April 21, 2011 @11:37AM (#2471012)
Yeah, usually they are on our blog right after the downtime, or in /r/announcements on reddit.
Re:Amazon Web Services are down by masklinn on Thursday April 21, 2011 @07:50AM (#2470259)
Or have a fallback, maybe?
Re:Amazon Web Services are down by jedberg on Thursday April 21, 2011 @06:26AM (#2470062)
Yeah, I'm one of the reddit admins.