News for hackers. Stuff that matters.
Re:Show HN: Hivemind – Distributed jobs using AWS Lambda fun... by tyingq on Thursday September 28, 2017 @10:12PM (#15362272)
There's an SQS/Lamba integration. Push 10k messages in a queue as jobs. The lambdas pull them off and delete them once done.

https://cloudonaut.io/integrate-sqs-and-lambda-serverless-ar...

Re:Show HN: Hivemind – Distributed jobs using AWS Lambda fun... by teej on Friday September 29, 2017 @02:14AM (#15363138)
From what I can tell, this doesn't really help you know when your queued up lambdas have all completed, which is what GP wants.
Re:Show HN: Hivemind – Distributed jobs using AWS Lambda fun... by tyingq on Friday September 29, 2017 @07:46AM (#15364436)
You're right. I assumed the queue depth api call would be useable as a counter. It appears though, that it's only an approximate count unless you use FIFO queues.

I guess the individual Lambdas could pop an "I'm done" message onto a FIFO queue. With a single Lambda consumer of that queue that updates an S3 bucket with a count. Done when count == 10,000. That's atomic, and no polling, but feels a bit smelly.