It would be easily done by AWS SQS you can put as many elements to queue as you want (some limits apply, but may be easily lifted) and then remove items from the queue when the job is done, by tunning invisibility time-out you make work by your requirements. You can use lambdas (too short for your case) or ec2 or Fargate as your workes and it may scale up or down depending on loads. What is cool that you can create multiple shards if you could predict how long jobs would take so some could be done in lambda other using ec2 thus reducing costs.