Re:A collection of lock-free data structures written in stan...
by josephg on Thursday May 11, 2023 @03:02AM
(#35898734)
Thanks for the link! I read that about a decade ago, and its still a great read. That article makes it seem ridiculous to implement software any other way.
And yet, yesterday I watched a talk about how ridiculous it is to use mmap to implement a database:
And I ended up side tracked watching some talks from CMU about database page buffers and such.
Its weird how people with different backgrounds (OS / kernel people and database people) come to very similar problems in computing with very different perspectives, and they end up implementing very different systems as a result. And in each case, each community thinks the other way is basically wrong.
My understanding is that Linus Torvalds thinks O_DIRECT is a ridiculous flag that database people probably don't want. And from a database perspective, its crazy how difficult the linux kernel makes it to write high performance filesystem code that never corrupts data if the system crashes. fsync is a misshapen sledgehammer, and disk write barriers or IO completion events are totally missing from linux.
And yet, yesterday I watched a talk about how ridiculous it is to use mmap to implement a database:
https://db.cs.cmu.edu/mmap-cidr2022/
And I ended up side tracked watching some talks from CMU about database page buffers and such.
Its weird how people with different backgrounds (OS / kernel people and database people) come to very similar problems in computing with very different perspectives, and they end up implementing very different systems as a result. And in each case, each community thinks the other way is basically wrong.
My understanding is that Linus Torvalds thinks O_DIRECT is a ridiculous flag that database people probably don't want. And from a database perspective, its crazy how difficult the linux kernel makes it to write high performance filesystem code that never corrupts data if the system crashes. fsync is a misshapen sledgehammer, and disk write barriers or IO completion events are totally missing from linux.