Re:Ask HN: Is your company sticking to on-premise servers? Why?
by lozenge on Thursday May 07, 2020 @04:30AM
(#23100937)
S3 is immediately consistent for new objects unless the service received a GET on the object before it was created. It's easy to use this to make an immediately consistent system.
[ Parent
| on HN ]
Re:Ask HN: Is your company sticking to on-premise servers? Why?
by ozkatz on Thursday May 07, 2020 @10:10AM
(#23103142)
S3 ListObjects calls are eventually consistent (i.e. list-after-put).
EMRFS [1] and S3Guard [2] mitigate this for data processing use cases.
[1] - https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-f... [2] - https://blog.cloudera.com/introducing-s3guard-s3-consistency...
Re:Ask HN: Is your company sticking to on-premise servers? Why?
by llarsson on Thursday May 07, 2020 @05:27AM
(#23101205)
Yes. HTTP POST (new objects) is immediate, HTTP PUT (updates to objects) is eventually consistent.
If you want to use this to create new objects all the time, rather than update ones you already have, you now have to keep track of which objects in your bucket are "old" and should no longer be there. But yes, totally doable.