>Then why comment?

Because the instruction list from above isn't backed with any solid reasoning and because commenting is what people do on HN.

>You clearly don't understand the use-case that AWS fits.

Pray tell, what is this enviable use case that I so clearly do not grasp?

>I've had jobs that took 18 hours to run on single machine finish in 12 minutes on Lambda. I could run that process 4 times a month and still stay within AWS's free tier limits.

Ok I'll bite. What takes 18 hours to run on a single machine but finishes in 12 minutes on Lambda.

I worked on a service a year ago that would stream a video from a source and upload it to a video hosting service. A few concurrent transfers would saturate the NIC. Putting each transfer job in a separate lambda allowed running any number of them in parallel, much faster than queuing up jobs on standalone instances

Yes but running multiple lambda jobs in parallel would still add upto more time than 12 minutes. What am I missing?

If I was running 10,000 transfer jobs in parallel and the longest of them took 12 minutes, the job would take 12 minutes

Yes but you are still charged for 18 hours of compute time?

That’s true, the cost needs to be factored into the model. But the near infinite bandwidth scalability allows the service to exist to begin with. If every job saturates your up and down bandwidth and takes 10 minutes, and you have 100 coming in a minute, you would need to design a ridiculous architecture that could spin up and down instances and handle queuing on the scale of thousands based on demand. Or you can write a simple lambda function that can be triggered from the AWS sdk and let their infrastructure handle the headache. I’m sure a home grown solution will become more cost effective at a massive scale but lambda fits the bill for a small/medium project

If you throw more resources at a bottlenecked problem, it will go faster.

Right, but without the lambda infrastructure it would be infeasible from infrastructure and cost perspective to spin up, let’s say 10,000 instances, complete a 10 minute job on each of them, and then turn them off to save money, on a regular basis

Isn't that also possible with EC2? Just set the startup script to something that installs your software (or build an AMI with it). Dump videos to be processed into SQS, have your software pull videos from that.

You'd need some logic to shut down the instances once it's done, but the simplest logic would be to have the software do a self-destruct on the EC2 VM if it's unable to pull a video to process for X time, where X is something sensible like 5 minutes.

We developed a web-based tool that described water quality based on your location. We generated screenshots of every outcome so the results could be shared to FB/titter. It was something on the order of 40k screenshots. Our process used headless chrome to generate a screenshot then it was uploaded to S3 for hosting.

Doing that in a series took forever. It took something like 14 hours to generate the screenshots, then 4 hours to upload them all. Spreading that load across lambda functions allowed us to basically run the job in parallel. Each individual lambda process took longer to generate a screenshot than on our initial desktop process, but the overall process was dramatically faster.

The parallelism argument doesn’t pass muster because you can do the same thing with a cluster of free tier t2.micro machines with any good orchestration platform, not just lambda.

This argument is basically: no counterpoint to the original post, but you can do things that are also easy on any other comparable platform.

Tell me again what I don’t understand?

> you can do the same thing with a cluster of free tier t2.micro machines

Not if you already used your free tier. Lambda is X free per month. Free tier t2.micro is X free for first 12 months.

> Tell me again what I don’t understand?

As someone who has done both, it's far, far easier to stand up a lambda than it is to manage a cluster of servers.

This still doesn’t make sense. There are portable systems that do the same, and have fully managed options, such as kubernetes.

In my mind the thing that makes lambda “easier” is they make a bunch of decisions for you, for better or worse. For real applications probably for the worse. If you have the knowledge to make those decisions for yourself you’re probably better off doing that.

> This still doesn’t make sense. There are portable systems that do the same, and have fully managed options, such as kubernetes.

The whole value proposition behind AWS is that they can do it better than your business due (directly or indirectly) to economies of scale. I think Kubernetes is super cool, but rebuilding AWS on top of Kubernetes is not cost effective for most companies--they're better off using AWS-managed offerings. Of course, you can mix and match via EKS or similar, but there are lots of gotchas there as well (how do I integrate Kubernetes' permissions model with IAM? how do I get Kubernetes logs into CloudWatch? how do I use CloudWatch to monitor Kubernetes events? etc).

An unoptimized query?

Why would the processing time differ? Would you have multiple lambdas running different subsets of the unoptimized query?

You could achieve the same with basically any decent concurrency model on a single machine.

Maybe one lambda is timing out to fallback, slowing sagas down. I don't know.