Re:But how, exactly, do databases use mmap?
by eqvinox on Saturday January 23, 2021 @01:14PM
(#25884700)
That's what msync() is for.
[ Parent
| on HN ]
Re:But how, exactly, do databases use mmap?
by cperciva on Saturday January 23, 2021 @03:12PM
(#25885613)
If you're tracking what needs to be flushed to disk when, you might as well just be making explicit pwrite syscalls.
Re:But how, exactly, do databases use mmap?
by monocasa on Saturday January 23, 2021 @01:21PM
(#25884769)
Right, but it can sync arbitrary ranges sooner, which is also awful for consistency.
Re:But how, exactly, do databases use mmap?
by reader_mode on Saturday January 23, 2021 @02:08PM
(#25885145)
Shouldn't your write strategy be resilient to that kind of stuff (eg. shutdown during a partial update) ?
Re:But how, exactly, do databases use mmap?
by gmueckl on Saturday January 23, 2021 @02:36PM
(#25885344)
Don't you need exact guarantees on write ordering to achieve that?
Re:But how, exactly, do databases use mmap?
by jorangreef on Saturday January 23, 2021 @02:53PM
(#25885480)
Yes, for almost all databases, although there was a cool paper from the University of Wisconsin Madison a few years ago that showed how to design something that could work without write barriers, and under the assumption that disks don't always fsync correctly:
"the No-Order File System (NoFS), a simple, lightweight file system that employs a novel technique called backpointer based consistency to provide crash consistency without ordering writes as they go to disk"
Re:But how, exactly, do databases use mmap?
by vlovich123 on Saturday January 23, 2021 @03:45PM
(#25885891)
Does that generalize to databases? My understanding is that file systems are a restricted case of databases that don’t necessarily support all operations (eg transactions are smaller, can’t do arbitrary queries within a transaction, etc etc).
Re:But how, exactly, do databases use mmap?
by jorangreef on Sunday January 24, 2021 @12:09AM
(#25889476)
I think it would, because if your underlying file system can provide crash consistency then that can be leveraged as a primitive by the database.
Re:But how, exactly, do databases use mmap?
by bonzini on Saturday January 23, 2021 @02:48PM
(#25885433)
You can do write/sync/write/sync in order to achieve that. It would be nicer to have FUA support in system calls (or you can open the same file to two descriptors, one with O_SYNC and one without).