News for hackers. Stuff that matters.
Re:Feeding data to 1000 CPUs – comparison of S3, Google, Azu... by toomuchtodo on Tuesday January 05, 2016 @05:29PM (#10846578)
RPS to S3 is limited, but not throughput to S3, except by bucket. Higher throughput can be achieved by sharding your data across multiple buckets. Also, its important to properly namespace your keys within buckets to ensure its efficiently distributed across underlying data partitions.
Re:Feeding data to 1000 CPUs – comparison of S3, Google, Azu... by jedberg on Tuesday January 05, 2016 @05:35PM (#10846619)
Unless that is a semi-recent change, that is not what I've been explicitly told. To be fair my information is at least two years old now.
Re:Feeding data to 1000 CPUs – comparison of S3, Google, Azu... by toomuchtodo on Tuesday January 05, 2016 @05:36PM (#10846628)
My experience is solely based on recent production workloads attempting to pull TBs of data out of S3 very quickly to restore data to less than reliable indexed datastore. YMMV.