Re:How Far Out Is AWS Fargate?
by jasonjei on Thursday August 15, 2019 @11:06AM
(#20705854)
Wouldn’t it be better to break your tasks into smaller, discrete units instead of a 15 minute job, such as using a message queue (SQS)?
[ Parent
| on HN ]
Yes, that's ideal. But:
* Sometimes it's not efficient to break a task down or it simply can't be done. * Sometimes it just makes the code hard to maintain if breaking it down into different components is a difficult task
An example I can think of is loading a very large CSV file into a database. Lambda doesn't have enough memory to read the whole file and even if it did, it might take more than 16 minutes to run. I can try to break it out into multiple tasks and it'll probably be faster, but reading that code is (generally) gonna be much harder.
If your unit of work is a transaction that takes 15+ minutes that's not possible. A common example is nightly ETL jobs that update the world.
Re:How Far Out Is AWS Fargate?
by notyourwork on Thursday August 15, 2019 @11:50AM
(#20706258)
That’s not a good use case for lambda.
Well duh. That's why the top level comment is saying, "That's where Fargate comes in...".
Re:How Far Out Is AWS Fargate?
by notyourwork on Thursday August 15, 2019 @07:33PM
(#20710734)
> If your unit of work is a transaction that takes 15+ minutes that's not possible. A common example is nightly ETL jobs that update the world.
So what are you suggesting here?