Re:Downloading files from S3 with multithreading and Boto3
by vtuulos on Sunday April 11, 2021 @02:09PM
(#26772212)
re: separate process - fault-tolerance is a key requirement. There are a myriad of ways how a highly parallelized, network- and data-intensive code can fail, so isolating it in a separate process is a safer approach than trying to try-except everything and hope it works.
Good catch re: the warning about consistency! The docs were written before the change :)
Good catch re: the warning about consistency! The docs were written before the change :)