I considered citing that in my post, but it didn't fit anywhere conveniently. In general I like the New Jersey approach of "push complexity to the caller", but not in cases like this where it's impossible for the caller to handle the complexity correctly.

I have the same issue with eventual consistency, for what it's worth: I'm fine with the fact that my updates might not be globally visible immediately, but I want to have a way that at some point I can know that they have propagated (aka. "eventually known consistency").

The problem with "push complexity to the caller" is that there are a lot more callers than implementors, and to make non-buggy code you have to educate callers who

- may not have time ("my boss told me to ship this yesterday")

- may not know ("I couldn't afford a copy of the standard, and I got my documentation by browsing web pages")

- may not understand ("WTF is this extra thing I have to worry about?")

In short, a perfect recipe for making a world of software that mostly works, most of the time, but is full of bugs in edge conditions and under stress.

I don't write code like that. It's never fun to explain to a customer why your stuff falls apart under load, for instance.

The solution is layering: OS-level APIs can push complexity to the caller, but the "caller" can be a library that implements a higher-level API and hides this complexity from the application. Now you have the best of both worlds: people who want to do simple things use higher-level APIs but people who have more sophisticated needs can use the lower-level API directly.

If the lowest-level API does complicated things under the covers, then the power user is left with no recourse when the "one size fits all" API didn't fit.

Yes, exactly. I don't understand why the Unix guys didn't solve this in libc, where it's both easy and simple (provided the system call returns all the information needed, as it surely would have been made to do had this approach been taken).

Sure, occasionally someone will need to issue the bare system call without the libc wrapper, but that could have a different name, e.g. '_close'.

Is eventually known consistency even possible if at any point while the notification of consistency is being transferred to you, the system can enter a non/eventual-consistent state again?

By "eventually known consistency" I don't mean "at some point I will know that the system is consistent"; rather, I mean "at some point I will know that the system is consistent with respect to operation X" (or alternatively "... with respect to all operations performed before time T").

There's actually two distinct things you might want to know:

1) The current state of the system.

2) Whether the current state of the system is equal to or later than one reflecting your change.

#1 is technically unknowable without a locking mechanism, but #2 is theoretically possible. Assuming reliable time synchronization, your worst-case scenario would be a timestamp on every record, and checking the timestamp on each node (you can do this with Cassandra, for example).