Re:Show HN: Hivemind – Distributed jobs using AWS Lambda fun...
by teej on Friday September 29, 2017 @02:14AM
(#15363138)
From what I can tell, this doesn't really help you know when your queued up lambdas have all completed, which is what GP wants.
Re:Show HN: Hivemind – Distributed jobs using AWS Lambda fun...
by tyingq on Friday September 29, 2017 @07:46AM
(#15364436)
You're right. I assumed the queue depth api call would be useable as a counter. It appears though, that it's only an approximate count unless you use FIFO queues.
I guess the individual Lambdas could pop an "I'm done" message onto a FIFO queue. With a single Lambda consumer of that queue that updates an S3 bucket with a count. Done when count == 10,000. That's atomic, and no polling, but feels a bit smelly.
https://cloudonaut.io/integrate-sqs-and-lambda-serverless-ar...