News for hackers. Stuff that matters.
Re:Serverless Map/Reduce by ronack on Friday November 04, 2016 @02:01PM (#12874544)
We had it raised to several thousand concurrent requests with no questions asked. YMMV based on age of account, amount of spend, etc.
Re:Serverless Map/Reduce by ceejayoz on Friday November 04, 2016 @02:04PM (#12874571)
Yeah, most of the limits are there just to prevent "whoops, we fired up 500 $2/hour servers and left them running all month, can you refund us?" sort of newbie situations. Very few are technological and AWS has been happy to bump them up for me in each occasion I've had a need.
Re:Serverless Map/Reduce by vidarh on Friday November 04, 2016 @02:53PM (#12874943)
I've had plenty of cases where it's taken time, and required escalation and justification to get even relatively modest instance limits raised, depending on the data centre and instance types.

It's not technological - it's about ensuring they have a reasonable handle on capacity and what they happen to have plenty of.

It's not a problem as long as you're not asking for astronomical limits, but people need to be aware of it and request increases in advance of starting to hit limits rather than assume it'll be near immediate to get them lifted.

Re:Serverless Map/Reduce by teej on Friday November 04, 2016 @02:09PM (#12874613)
Some are, some aren't. For example, my experience has been that it's harder to get the 100 S3 bucket-per-account limit raised.
Re:Serverless Map/Reduce by jedberg on Friday November 04, 2016 @02:13PM (#12874653)
That used to be a hard technological limitation but they fixed it a few months back. You should be able to get more now (but you have to have a good justification for it).
Re:Serverless Map/Reduce by chrisseaton on Friday November 04, 2016 @02:39PM (#12874842)
100 seems an odd number to be a technological limitation, doesn't it? It's not a power of two or anything so it's not the size of an integer. How could technology have been limiting it?
Re:Serverless Map/Reduce by sharpy on Friday November 04, 2016 @02:56PM (#12874977)
S3 bucket names are globally scoped - ie. they are literally scarce resources.
Re:Serverless Map/Reduce by jedberg on Friday November 04, 2016 @04:03PM (#12875598)
Technology in the sense that it was a hard coded variable, but it was picked arbitrarily.
Re:Serverless Map/Reduce by ma2rten on Saturday November 05, 2016 @12:40AM (#12878212)
What would be a good justification? Is there anything you can do with 101 buckets that you can't do with folders one bucket?
Re:Serverless Map/Reduce by ceejayoz on Sunday November 06, 2016 @11:47AM (#12885883)
Requester-pays billing (http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketBilling...) for more than a hundred clients, static sites served through CloudFront, etc.
Re:Serverless Map/Reduce by ceejayoz on Friday November 04, 2016 @02:47PM (#12874903)
IIRC that was one of the few the AWS docs specifically called out as unraisable.
Re:Serverless Map/Reduce by Jdam on Saturday November 05, 2016 @12:04PM (#12880303)
I once wanted to raise the limit of g2 instances from 2 to 100 in us-east-1 and the reply from the customer agent was he first "has to check if that does not affect overall region stability" WTF Anyway, it was raised a couple of days later.
Re:Serverless Map/Reduce by teej on Friday November 04, 2016 @02:05PM (#12874582)
Very useful, thanks!