Re:Fsyncgate: Errors on fsync are unrecoverable (2018)
by mjw1007 on Sunday July 21, 2019 @01:28PM
(#20492457)
The annoying thing is that the Postgres people don't even really need "sync this file right now" for the main data files.
AIUI what they'd ideally like is "sync this file reasonably soon, and let me know when it's finished".
Re:Fsyncgate: Errors on fsync are unrecoverable (2018)
by the8472 on Sunday July 21, 2019 @04:58PM
(#20493624)
io_uring supports issuing async sync_file_range requests, which can be used to do most of the heavy lifting.
You will still need a final fsync to also get the metadata written, but that should faster at that point.
Re:Fsyncgate: Errors on fsync are unrecoverable (2018)
by the8472 on Tuesday July 23, 2019 @03:26PM
(#20509473)
correction: I took a closer look at liburing and it also supports straight fsync[0], not just sync_file_range, this makes it even simpler.
[0] http://git.kernel.dk/cgit/liburing/tree/src/io_uring.h?id=90...
Re:Fsyncgate: Errors on fsync are unrecoverable (2018)
by cyphar on Sunday July 21, 2019 @08:42PM
(#20494640)
It should be noted that io_uring landed (in 5.1) a fair while after the whole Postgres back-and-forth started in 2018.
Re:Fsyncgate: Errors on fsync are unrecoverable (2018)
by masklinn on Monday July 22, 2019 @02:07AM
(#20495672)
It should also be noted that even sync_file_range's manpage warns you against using it, which tells you how reliable that thing is.
Re:Fsyncgate: Errors on fsync are unrecoverable (2018)
by the8472 on Monday July 22, 2019 @03:59PM
(#20501733)
It's not really that you shouldn't use it because it is dangerous in itself. It's just doesn't do what one would naively expect it to do. It's still useful as "flush dirty pages" hinter while preparing the real fsync.
Now imagine asking him to support being thrown balls randomly, at any interval. He may make it work, but I'd imagine he will stutter a bit.
In my experience, anytime you interrupt the page cache's normal routine, it stutters everything. I've seen the "sync" command freeze my Ubuntu machine (music player, GUI, etc).
I work on embedded devices, and my employer wanted to reduce the window at which data-loss would happen for 30mb+ files (video capture). It wasn't a supported use case, "But why not! It makes our product theoretically better!" I put my foot down. We aren't touching page-cache until there is a clear benefit to the user. It almost got me fired, but good riddance if so.