Re:LiteFS a FUSE-based file system for replicating SQLite
by dan353hehe on Tuesday July 26, 2022 @04:18PM
(#32243420)
I’m curious, was there a specific reason that FUSE was used instead of the FVS layer inside to SQLite itself? I realize that it would mean that a custom module would have to be loaded, and maybe that was undesirable. Full disclosure I have not read the link yet as I am a little pressed for time, but I will later tonight when I have time. If it’s mentioned in the readme, feel free to ignore me :) I ask because I have been building a VFS for SQLite to handle multiple distributed writers and replicating data blocks around to different places.
Re:LiteFS a FUSE-based file system for replicating SQLite
by benbjohnson on Tuesday July 26, 2022 @04:41PM
(#32243697)
It's a good question. It's mainly about reducing friction for the end user. I wrote up a related response here: https://news.ycombinator.com/item?id=32243305
The FUSE layer does have performance trade-offs. Read queries aren't affected as much since the OS page cache is in the kernel and avoids the FUSE layer entirely. Most databases are under 10GB (or honestly probably less than 1GB) so you can fit most of that in the page cache for any moderately sized VM. The write side will incur a performance hit but I haven't taken benchmarks yet. There's a lot that can still be optimized around that.
The biggest help that I could use is just to get feedback on how it feels to use and any ways I can improve it. The end goal is to make it ridiculously simple to spin up and "just work" so any ideas towards that goal would be awesome.