News for hackers. Stuff that matters.
Re:Feeding data to 1000 CPUs – comparison of S3, Google, Azu... by rmcpherson on Tuesday January 05, 2016 @07:35PM (#10847363)
In S3 tests on c3.8xlarge instances, I've seen 8 Gbps throughput on both uploads and downloads using parallelized requests. Testing with iperf between two of the same instances maxed out about 8 Gbps as well so the throughput limitation is likely EC2 networking rather than S3.

These tests were done over a year ago so bandwidth limitations on EC2 may have changed since.

This testing was with https://github.com/rlmcpherson/s3gof3r

Re:Feeding data to 1000 CPUs – comparison of S3, Google, Azu... by zbjornson on Tuesday January 05, 2016 @07:40PM (#10847389)
That's really cool. Wonder if the same technique (parallel streams) would help for Azure and GCS. I know GCS has some built-in capabilities for composite uploads/downloads, which might achieve a similar effect.