News for hackers. Stuff that matters.
Re:How can you trust a disk to write data? by generalizations on Tuesday February 22, 2022 @10:04AM (#30428774)
> This employs a directory used for temporary writes to unique filenames, followed by a move into place in a 'real' directory, where files should be accessible once write is deemed to be complete.

This still seems like it makes the assumption that the move will complete. Or is 'mv' significantly different from the full write?

Re:How can you trust a disk to write data? by pkhuong on Tuesday February 22, 2022 @10:12AM (#30428890)
The move (rename) is a metadata-only operation and is atomic on POSIX-compliant platforms and in reality.
Re:How can you trust a disk to write data? by yakubin on Tuesday February 22, 2022 @10:52AM (#30429409)
Rename being atomic doesn't imply that after a rename completes, it's flushed to cold storage. It just implies that a rename is either completely flushed to cold storage, or the data in cold storage looks as if there was never a rename.

Also, the data you wrote to a file before the rename may not have been flushed completely to cold storage. Rename doesn't change here anything. The data may still be served from the disk controller's cache.

The rename trick is mostly useful for "committing" long operations, which even on the high level are not atomic. But it has nothing to do with flushing.

Also, renaming a file is a filesystem-level operation. From the POV of a disk, it looks the same as any other write. So if you can't make sure that normal writes are flushed, you can't make sure that renames are flushed.

Another note: rename(2) used not to be even atomic wrt power loss on ext4 (which was later fixed), so it's not guaranteed. POSIX itself only guarantees atomicity wrt other filesystem operations. Atomicity against power loss is a distinct checkbox. You need to check your specific filesystem.

Re:How can you trust a disk to write data? by cout on Tuesday February 22, 2022 @11:28AM (#30429846)
I'll add this: on spinning disks it's normal to reorder write operations; it is optimal to order writes in a way that matches the layout on disk. That way, each sector is written as the write head passes over the sector. We like to think of a disk-backed filesystem as slow, persistent RAM, but in reality it's a lot more complex than that.
Re:How can you trust a disk to write data? by generalizations on Tuesday February 22, 2022 @10:17AM (#30428959)
Sure; IIRC, it only changes the inode. However, it's still trying to change bits on the disk, regardless if those are data or metadata. Can the metadata-only bits that the disk claims to write in order to implement the move be trusted to have been written?
Re:How can you trust a disk to write data? by steerablesafe on Tuesday February 22, 2022 @10:22AM (#30429027)
It might be atomic on the OS level (no processes can see an "intermediate" state), but it might not be atomic when you yank the power cord. POSIX in general doesn't offer any guarantees there, AFAIK.