Wouldn’t a lambda be far cheaper?
Re:New EC2 Instance Types of re:Invent 2021
by cldellow on Sunday December 05, 2021 @07:10PM
(#29454673)
In my experience, no. Quite the opposite.
Lambdas can be cost effective for bursty loads, but if you can push sustained CPU utilization, as this example does, EC2 is cheaper.
For example, an ARM-based Lambda with 2 GB of RAM running for 1 hour is $0.09. An a1.medium is $0.025/hr on-demand or $0.0084/hr spot.
Part of this is that Lambda scales your CPU performance with your memory usage. So even if your code can run in 128 MB of RAM, there's no cost benefit to you if it's CPU-constrained, because it will run 16x slower than if you allocated it 2048 MB of RAM.
My framework runs a fairly simple pipeline: for each 1GB file, stream it from S3, decompress it, search it with a regex library, stream results back to S3.
A1 spot instances have consistently been cheaper for this task when measured on a $/GB scanned basis. Part of this might be that the framework doesn't need lots of RAM, so paying for more than 2GB per core on the non-A1 instances is just wasted expense for me.