News for hackers. Stuff that matters.
Re:Downloading files from S3 with multithreading and Boto3 by emasquil on Saturday April 10, 2021 @09:36PM (#26766847)
In this case I just wanted to keep it simple and do it the "native" way. I've been hearing a lot about aioboto3 and I'll surely check it out. However, it seems to have some limitations like https://github.com/boto/botocore/issues/458
Re:Downloading files from S3 with multithreading and Boto3 by ben509 on Saturday April 10, 2021 @10:45PM (#26767123)
That issue is just botocore not supporting async, which aiobotocore addresses. It's also very long; is there a particular comment on that issue that you wanted to point out?
Re:Downloading files from S3 with multithreading and Boto3 by emasquil on Sunday April 11, 2021 @01:45PM (#26772060)
I just mentioned the issue because a colleague of mine did some quick tests with aioboto3 and he told me it was downloading the files sequentially. We thought it might be related to that.

You made a great point about going async if you need a working command channel.