Re:S3 trickery: using it as a scheduler
by scarface74 on Friday September 14, 2018 @08:10AM
(#17986593)
Don’t blame Serverless. Blame it on a poor implementation. AWS Cloudwatch supports cron rules that include the year that it should be scheduled and it can trigger lambdas directly. There was no reason he had to over complicate this.
As far as using a queue, lambda also supports triggering based on a queue.
Don't forget that the above includes multi-datacenter, multi-region redundancy / failover. Could be important sometimes.
Waking on items in queue would be better, though, and AWS has it built-in; the authors just did not use it for some reason.
Not saying it's right or wrong solution, but saying that a simpler solution would be to take a server and put on it a job scheduler process is misleading. People shouldn't forget that it never ends with a single server and a single process, you have to manage it to make it production ready and fault tolerant.
Sometimes doing the extra step to make it serverless compatible is worth the effort.
Re:S3 trickery: using it as a scheduler
by pastununtrium on Friday September 14, 2018 @07:23AM
(#17986346)
Agreed, lol. I’d say let Tech Darwinism take its toll on the weak.
Instead of doing the simple thing and running a server process that periodically checks some queue for tasks to execute, this "scheme" involves triggering a lambda function every minute, to do some IO operations on a distributed file store where the execution date metadata is embedded in the filenames to see if there is some task to execute. If there is, some more distributed IO operations are required to move the file to another part of the distributed file storage, where some listener will notice a new file was added and trigger the actual lambda function with the task-specific code...
But hey, you're not running any servers yourself...