Perhaps we need database and backup oriented filesystems and not funny looking files on top of generic filesystems.
There are some additional performance and behavioral advantages to working with the storage devices directly. Anecdotally, if you run databases on virtual machines (never recommended but many people do), using raw block devices instead of a file system often seems to eliminate much of the disk I/O weirdness that occurs under VMs.
e.g. in mysql: https://dev.mysql.com/doc/refman/8.0/en/innodb-raw-devices.h...
What was surprising is that the direct I/O behavior appears to be conditional on whether you are accessing the storage through a file system. My database kernel is block device agnostic, using files and raw devices interchangeably via direct I/O. Against expectations, when we accessed the same virtualized storage as raw block devices, the behavior was like bare metal even though we are running the exact same operations over the same direct I/O interface in a VM. Basically, the only difference was the file descriptor type.
I'm guessing that file systems are virtualization aware to some extent and access through them is actively managed; raw device accesses are VM oblivious and simply passed through by the storage virtualization layer.
On top of that, many issues you may be facing re. files now have already been resolved if you change the stack: you can't do transactions with fs.
Proper handling and reporting of hardware-level errors all the way up through the stack (driver, block layer, filesystem, C library) to the application so it can recover in a reliable way is not a simple operation!
Simple operations are open/close/read/write. Those work. Until they don't, then you need to know how far back the operations you already did and "assumed" had worked didn't. And in this case the promise made to PostgreSQL by fsync() wasn't as firm as the "obvious" interpretation of the documentation would lead one to believe.
I'm certainly not qualified to criticize anyone for the current situation, and, as the article points out, even some of the more egregious sounding behavior (marking as clean pages after writing fails) has a pretty reasonable explanation. But, IIRC, as storage capacities continue to rise, error rates aren't falling nearly as fast. So, I'm left kinda wondering if there is some day in the future where the likely hood of encountering an error finally gets high enough that things don't work pretty well anymore.