News for hackers. Stuff that matters.
Re:Serverless Map/Reduce by ralusek on Friday November 04, 2016 @01:39PM (#12874381)
This is a screenshot of my google search from 2 days ago:

http://i.imgur.com/BNAcSsn.png

I've been using Lambda quite a bit, I think it's SO amazingly useful. Tasks that are highly parallelized and CPU intensive can literally be infinitely scaled out. I find it weird that their poster child use case is still always a reactive event like watching S3 and formatting images. There are so many use cases for directly invoking a lambda directly from your code.

Imagine a case where you had to parse a million documents with a relatively expensive computation, let's say 250MS per document. Maybe you have a solid machine with a few cores that's running your server, but even then you can't have the server cpu locked for so long, so naturally you'd need some sort of worker server set up. With a good machine and multiple cores, maybe you get 8 running at once. With a lambda, you can forego the worker server altogether. Just invoke a million lambdas directly from your application server, completely parallelized.

Theoretically, you've taken something that would take 70 hours and had it run in 250ms without having to set up any additional infrastructure.

Re:Serverless Map/Reduce by teej on Friday November 04, 2016 @01:48PM (#12874436)
I just restructured a job to be parallel and run on Lambda and I couldn't be happier. Glad to see this idea is gaining mindshare. I found that it was easy to test, easy to debug, easy to maintain, not very expensive, and just kinda worked.

The only caveat for those considering taking this path - AWS has an account-wide limit of 100 concurrent lambdas running at once. There's no way to know how many are currently running unless you keep track of it yourself. You'll only find out when you try to kick off a job and it get's throttled. I haven't contacted support yet to find out how hard it is for the limit to get raised or what they'll raise it to.

Re:Serverless Map/Reduce by ronack on Friday November 04, 2016 @02:01PM (#12874544)
We had it raised to several thousand concurrent requests with no questions asked. YMMV based on age of account, amount of spend, etc.
Re:Serverless Map/Reduce by ceejayoz on Friday November 04, 2016 @02:04PM (#12874571)
Yeah, most of the limits are there just to prevent "whoops, we fired up 500 $2/hour servers and left them running all month, can you refund us?" sort of newbie situations. Very few are technological and AWS has been happy to bump them up for me in each occasion I've had a need.
Re:Serverless Map/Reduce by vidarh on Friday November 04, 2016 @02:53PM (#12874943)
I've had plenty of cases where it's taken time, and required escalation and justification to get even relatively modest instance limits raised, depending on the data centre and instance types.

It's not technological - it's about ensuring they have a reasonable handle on capacity and what they happen to have plenty of.

It's not a problem as long as you're not asking for astronomical limits, but people need to be aware of it and request increases in advance of starting to hit limits rather than assume it'll be near immediate to get them lifted.

Re:Serverless Map/Reduce by teej on Friday November 04, 2016 @02:09PM (#12874613)
Some are, some aren't. For example, my experience has been that it's harder to get the 100 S3 bucket-per-account limit raised.
Re:Serverless Map/Reduce by jedberg on Friday November 04, 2016 @02:13PM (#12874653)
That used to be a hard technological limitation but they fixed it a few months back. You should be able to get more now (but you have to have a good justification for it).
Re:Serverless Map/Reduce by chrisseaton on Friday November 04, 2016 @02:39PM (#12874842)
100 seems an odd number to be a technological limitation, doesn't it? It's not a power of two or anything so it's not the size of an integer. How could technology have been limiting it?
Re:Serverless Map/Reduce by sharpy on Friday November 04, 2016 @02:56PM (#12874977)
S3 bucket names are globally scoped - ie. they are literally scarce resources.
Re:Serverless Map/Reduce by jedberg on Friday November 04, 2016 @04:03PM (#12875598)
Technology in the sense that it was a hard coded variable, but it was picked arbitrarily.
Re:Serverless Map/Reduce by ma2rten on Saturday November 05, 2016 @12:40AM (#12878212)
What would be a good justification? Is there anything you can do with 101 buckets that you can't do with folders one bucket?
Re:Serverless Map/Reduce by ceejayoz on Sunday November 06, 2016 @11:47AM (#12885883)
Requester-pays billing (http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketBilling...) for more than a hundred clients, static sites served through CloudFront, etc.
Re:Serverless Map/Reduce by ceejayoz on Friday November 04, 2016 @02:47PM (#12874903)
IIRC that was one of the few the AWS docs specifically called out as unraisable.
Re:Serverless Map/Reduce by Jdam on Saturday November 05, 2016 @12:04PM (#12880303)
I once wanted to raise the limit of g2 instances from 2 to 100 in us-east-1 and the reply from the customer agent was he first "has to check if that does not affect overall region stability" WTF Anyway, it was raised a couple of days later.
Re:Serverless Map/Reduce by teej on Friday November 04, 2016 @02:05PM (#12874582)
Very useful, thanks!
Re:Serverless Map/Reduce by wnevets on Friday November 04, 2016 @05:13PM (#12876171)
raising the limit is a pretty easy request to aws support.
Re:Serverless Map/Reduce by brianwawok on Friday November 04, 2016 @06:22PM (#12876667)
So google compute setup I did a while back with preemptible instances + a celery queue + some autoscale based on load...

The guts to make all that work was 50 or so lines of config. I think my auto scale script was 20 lines or so of Python.

I guess the biggest downside was spinning up the new server took about 2 minutes, so for big load spikes it took a bit for it to level out... but with GCE per minute billing, all the napkin math I did says it is a fair bit cheaper per cpu unit to do it this way.

In conclusion, someone good with devops + a normal work queue system could do this years ago. I guess it's cool to lower the barrier to entry, but not it does not seem like a game changer. It totally IS cool to scale out backend work to a job farm. Just seems like not the only way to do it.

Re:Serverless Map/Reduce by vgt on Friday November 04, 2016 @07:17PM (#12877036)
That's essentially what Google Cloud Dataproc gives you (managed Hadoop/Spark):

- Per-minute billing

- 0-to-cluster in under 90 seconds (aim for 30 seconds)

- Pre-emptiblem VMs

- Custom VMs

Now you start with a job, pay a 30 second penalty, and execute it on an entirely ephemeral cluster. The "get a cluster and fill it with jobs and round up to an hour" model is indeed outdated IMHO.

(work at Google Cloud)

Re:Serverless Map/Reduce by brianwawok on Saturday November 05, 2016 @08:31AM (#12879479)
Well it depends on my workload. The setup above on GCE had a stable load of X thousand jobs per minute, then burst loads up to 100x for short times. So for me it made sense to have a 24/7 celery cluster for the base load and add and remove nodes for the variable node. There was never a point shutting down the cluster made sense.
Re:Serverless Map/Reduce by edblarney on Saturday November 05, 2016 @01:07AM (#12878282)
" I find it weird that their poster child use case is still always a reactive event like watching S3 and formatting images."

Because those are the things that need to be done in the real world, 99% of the time.

Sadly, we don't all get to work on cool stuff :)

But I think as more people get access to the power and find uses cases, it will get used more.

The 'next' step for Lambda is serverless hosting for web-sites, something which Lambda is 'almost but not quite' ready for, the limiting factor being transactional latency, especially in some weird cases. Once they have that sorted out, I will never want to see a server again :)

Re:Serverless Map/Reduce by PetahNZ on Saturday November 05, 2016 @05:08AM (#12878869)
Except for the fact Lambda is limited to 100 concurrent executions.
Re:Serverless Map/Reduce by shawn-butler on Sunday November 06, 2016 @04:47AM (#12884322)
This isn't a fact. The default limit is 100 concurrent per region.

Also a simple conversation with AWS increases the limit[0].

They obviously don't want people to write absurd runaway structures and 100 is an arbitrarily chosen value to guard against that.

[0]: https://aws.amazon.com/lambda/faqs/

Re:Serverless Map/Reduce by hueving on Friday November 04, 2016 @01:50PM (#12874450)
>Theoretically, you've taken something that would take 70 hours and had it run in 250ms without having to set up any additional infrastructure.

And you've spent the cost of building out that 8 server infrastructure in one batch.

Re:Serverless Map/Reduce by etrain on Friday November 04, 2016 @01:59PM (#12874528)
One of the nice things about lambda is the billing is super granular - you get billed at 100ms intervals.

Assuming 70 hours at $0.000000834/100ms [1]

The whole job costs $2.10.

[1] - https://aws.amazon.com/lambda/pricing/

Re:Serverless Map/Reduce by nostrademons on Friday November 04, 2016 @03:35PM (#12875333)
By comparison (since I was curious), 70 hours on an m3.medium spot instance will run around $0.70. On an on-demand, it's about $5.39. EMR will cost you about $7.00 on top of the EC2 costs.

If you can peg the CPU and don't mind getting interrupted, spot instances are still a fair bit cheaper. But Lambda looks pretty attractive for any other use-case where the statelessness of Lambda doesn't bite you.

Re:Serverless Map/Reduce by etrain on Friday November 04, 2016 @04:06PM (#12875621)
Keep in mind that with EC2 you're billed hourly so the fastest a 70 cpu-hour job could finish on m3.medium for $0.70 is 1 hour, and that's ignoring setup time, etc.

Meanwhile, on Lambda, you can actually run 1600 60s jobs (or 27 CPU-hours) in 3 minutes. This is inclusive of setup time, job submission, stragglers, etc. [1]

Of course, if you've got sustained load, it's cheaper to go with spot instances, but the "occasionally I need a buttload of compute," model is well-served by Lambda.

[1] http://ericjonas.com/pywren.html

Re:Serverless Map/Reduce by IanCal on Saturday November 05, 2016 @07:35AM (#12879293)
As a note for people, if your constraints are a bit different then these are some services to check out:

Joyent Manta: https://www.joyent.com/manta

Hyper: http://hyper.sh

Possibly Joyent Triton: https://www.joyent.com/triton

I personally often want to run a bunch of things for ~1-15 minutes, and have too much data or setup to fit neatly in a lambda function. However, I don't need 1000 things running simultaneously, although manta would help still there.

I'd love to see some more layers over the top of services like this, hopefully someday getting us back to picloud. I miss that service.

Re:Serverless Map/Reduce by marktangotango on Saturday November 05, 2016 @05:53PM (#12882108)
How secure are docker hosts like hyper.sh? I've always been skeptical, the multitenant docker security story hasn't been very encouraging, or has that changed?
Re:Serverless Map/Reduce by riobard on Sunday November 06, 2016 @12:44AM (#12883565)
hyper.sh containers are kernel-isolated like virtual machines.
Re:Serverless Map/Reduce by teej on Friday November 04, 2016 @02:07PM (#12874596)
70 * 60 * 60 * 10 * $0.000000834 = $2.1
Re:Serverless Map/Reduce by ralusek on Friday November 04, 2016 @02:13PM (#12874656)
As the other poster who replied to you showed, it's actually incredibly cheap. Additionally, it's perfectly elastic. That expensive hardware you provision or buy for your server that can handle this task (at a much, MUCH slower rate) would cost you significantly more. In addition to the cost of the server and running it, you also have a much harder job developing it. If we're talking about multicore, you're now managing the job concurrently on a single machine, and need the code to facilitate it.

I also feel like your comment is implying that the outcomes of the lambda option vs the worker server are comparable. As mentioned, the worker machine will still run in hours, rather than less than a second.

Re:Serverless Map/Reduce by teej on Friday November 04, 2016 @02:03PM (#12874559)
The cost to execute a 250ms Lambda on the most expensive tier 1M times is about $7.70. If your data is in S3 the cost of 1M GETs at the most expensive is $0.40. You don't pay for transfer between S3 and Lambda.
Re:Serverless Map/Reduce by qaq on Friday November 04, 2016 @06:23PM (#12876681)
if it's one of you will not be able to get the limit if you do this even at a rate of 1 req/sec you will be paying close to 1.4 million USD per month just for compute.