News for hackers. Stuff that matters.
Re:POSIX v. reality: A position on O_PONIES by throwaway09223 on Saturday November 25, 2023 @02:38PM (#38415988)
There are all kinds of cases where filesystems might issue an extraneous sync, but it is not atomic, merely ordered. You've switched from talking about one to talking about the other and they are not the same. This is the core of our disagreement.

Because it is not an issue of atomic behavior but merely ordering there's no reason to modify the kernel or touch syscalls. It would be far more reliable to provide this in a library -- rename(3) -- which would then automatically provide the benefit on every posix compatible filesystem.

It would also allow for a use case of wanting to rename a file without forcing a full data sync -- which could be performance critical behavior in some scenarios.

To recap:

No filesystems provide atomic syncing of data during rename(). The words don't even make sense. Ext doesn't do it, zfs doesn't do it, xfs doesn't do it. Achieving this would require mechanisms that don't exist.

Some filesystems do guarantee that an fsync will occur alongside a rename. There is no specific guarantee as to what file state will be synced. These filesystems do this to mask the impact of folks mis-using the API, paying an efficiency cost as a result.

In ext4, for example, the sync does NOT occur within the same transaction as the rename. The sync is NOT atomic (it would be wild if this were the case, as noted above -- architecturally inconsistent and extremely non-performant)

It would be ideal for this to happen in libc rather than in the kernel.

Re:POSIX v. reality: A position on O_PONIES by aidenn0 on Sunday November 26, 2023 @01:27PM (#38423721)
> No filesystems provide atomic syncing of data.

Yes everyone in the discussion agrees on this fact

> ...but it is not atomic, merely ordered. You've switched from talking about one to talking about the other and they are not the same

If the operations we are talking about are ordered, then this provides the atomicity that TFA is asking for. Properly ordering operations is a very common way of making a series of steps atomic.

Re:POSIX v. reality: A position on O_PONIES by throwaway09223 on Sunday November 26, 2023 @03:13PM (#38424572)
> "If the operations we are talking about are ordered, then this provides the atomicity that TFA is asking for."

No, it doesn't. I think we're reaching the heart of the disagreement here - around what "atomic" means.

Atomic means indivisible. It means that other things cannot happen at the same time - that the individual (possibly ordered) steps of an operation cannot be observed while in progress.

This is why it's absolutely nonsense to talk about atomicity across a crash.

The fundamental design of unix files and memory preclude this. It can't be done.

Re:POSIX v. reality: A position on O_PONIES by aidenn0 on Sunday November 26, 2023 @05:50PM (#38425894)
From another fork in this thread:

> ...the operation we wish to be atomic is 'replacing the mapping path A->data X with path A->data Y'.

The ordering makes this operation atomic! The fact that it might leave another file around (if a crash happens) is okay.

Re:POSIX v. reality: A position on O_PONIES by throwaway09223 on Sunday November 26, 2023 @08:00PM (#38426888)
What is it atomic with respect to?