News for hackers. Stuff that matters.
Re:A write-ahead log is not a universal part of durability by AdamProut on Monday July 01, 2024 @09:27AM (#40845671)
Maybe good to mention torn pages somewhere too? Both MySQL and Postgres jump through some hoops to both detect them and repair them [1][2]. So, even the scenario in the post where fsync is used to harden writes, the database still needs to handle torn pages (or requires using a file system \ storage that guarantees atomic page writes at the page size the database is using as several managed\cloud databases do).

[1] https://wiki.postgresql.org/wiki/Full_page_writes [2] https://dev.mysql.com/doc/refman/8.0/en/innodb-doublewrite-b...

Re:A write-ahead log is not a universal part of durability by eatonphil on Monday July 01, 2024 @10:12AM (#40846017)
Thanks Adam! I think torn writes would still be caught via checksums, no? Although that may be later than you'd wish.

I'm not confident but from reading that page it seems that for Postgres at least, if it did do checksums it might not need to count on page-level atomic writes?

Re:A write-ahead log is not a universal part of durability by AdamProut on Monday July 01, 2024 @10:19AM (#40846075)
Checksums can detect a torn page, but not always repair them. It's likely a good part of the database page is gone (i.e., an amount of data that matches the disk / file system atomic write unit size is probably missing). Torn page writes are a pretty common scenario too, so databases need to be able to fully recover from them - not just detect them and report a corruption (ie., just pull the power plug from the machine during a heavy write workload and you're likely to get one - it doesn't require a solar ray to flip a bit :) ).
Re:A write-ahead log is not a universal part of durability by eatonphil on Monday July 01, 2024 @10:23AM (#40846106)
That's fair. In the post I did mention disk redundancy (and I guess I only implied recovery) as one additional level for safety. Which I think is what you're getting at too.
Re:A write-ahead log is not a universal part of durability by mattashii on Monday July 01, 2024 @11:44AM (#40846909)
Disk redundancy won't help guarantee torn page protection if the writes across the redundant disks are not coordinated to have one start after the other finishes such that there is always have one copy of the page that is not currently being written. So writing to a RAID1 array won't help here without knowledge about how that raid1's writes work.