Re:Many small queries are efficient in SQLite
by Someone on Tuesday February 16, 2021 @05:42AM
(#26152722)
I don’t think you can atomically append more than one byte to files in unixes (the write call can return after having written some but not all requested bytes)
(Haven’t googled, but if that’s possible, I don’t see why write would have that limitation)
Re:Many small queries are efficient in SQLite
by LunaSea on Tuesday February 16, 2021 @07:37AM
(#26153305)
Interesting and confirmed in the write() syscall man pages. Thanks!
Do you have any other resources regarding these types of low level "gotchas"?
I remember PostgreSQL having such an issue two years ago for example.
Re:Many small queries are efficient in SQLite
by millstone on Tuesday February 16, 2021 @06:23AM
(#26152923)
Yeah, and eventually we reach the best-effort bedrock. Maybe the file is on a NFS mount, you call write(), it goes over the wire, who knows what happens!
Re:Many small queries are efficient in SQLite
by moosebear847 on Wednesday February 17, 2021 @07:25AM
(#26165584)
Dan luu's article mentioned these two options as unreliable. What's the reason for the disparity?
No, and anyone who says yes is lying. (Lockless NFS exists and is no fun.)
> Well, fundamentally it’s very hard to get it exactly right, and I imagine that’s why the implementation is a little involved
SQLite has set itself the horrible task of updating files in-place. I know of two reliable, simpler alternatives:
1. Appending to files through O_APPEND
2. Rewriting files through rename()
If SQLite has different magic syscalls then I would very much like to learn.