Re:Ask HN: Is your company sticking to on-premise servers? Why?
by llarsson on Thursday May 07, 2020 @12:01AM
(#23099586)
That S3 is eventually consistent with object updates (HTTP PUT) might also screw up things for a company whose core value is synchronized storage.
[ Parent
| on HN ]
Re:Ask HN: Is your company sticking to on-premise servers? Why?
by ayush--s on Thursday May 07, 2020 @03:48AM
(#23100734)
They might have implemented a metadata store 1 layer before S3 to guarantee read after write consistency since write to a new object is consistent in S3. Only updates are eventually consistent.
Re:Ask HN: Is your company sticking to on-premise servers? Why?
by dfsegoat on Thursday May 07, 2020 @12:35AM
(#23099765)
I don't mean to sound daft, just clarifying my own understanding, but isn't Dropbox eventually consistent (as a system)?
Re:Ask HN: Is your company sticking to on-premise servers? Why?
by llarsson on Thursday May 07, 2020 @02:54AM
(#23100416)
Oh, sure, but when they think they have written something to S3 and got a successful HTTP response back from the API, perhaps they want to be able to tell clients to go fetch the new data from the bucket. But those clients may not get the new data then, due to eventual consistency.
Re:Ask HN: Is your company sticking to on-premise servers? Why?
by lozenge on Thursday May 07, 2020 @04:30AM
(#23100937)
S3 is immediately consistent for new objects unless the service received a GET on the object before it was created. It's easy to use this to make an immediately consistent system.
Re:Ask HN: Is your company sticking to on-premise servers? Why?
by ozkatz on Thursday May 07, 2020 @10:10AM
(#23103142)
S3 ListObjects calls are eventually consistent (i.e. list-after-put).
EMRFS [1] and S3Guard [2] mitigate this for data processing use cases.
[1] - https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-f... [2] - https://blog.cloudera.com/introducing-s3guard-s3-consistency...
Re:Ask HN: Is your company sticking to on-premise servers? Why?
by llarsson on Thursday May 07, 2020 @05:27AM
(#23101205)
Yes. HTTP POST (new objects) is immediate, HTTP PUT (updates to objects) is eventually consistent.
If you want to use this to create new objects all the time, rather than update ones you already have, you now have to keep track of which objects in your bucket are "old" and should no longer be there. But yes, totally doable.
Re:Ask HN: Is your company sticking to on-premise servers? Why?
by boredpudding on Thursday May 07, 2020 @03:11AM
(#23100531)
Since the move to S3 is only 'a few years ago', my guess is this was not the reason for them to move away from S3 since service at Dropbox is pretty much the same.