Re:SQLite Disk Page Explorer
by ncruces on Saturday February 08, 2025 @11:48AM
(#42984205)
The problem with a block/page based solution (block per page is a bad idea, IMO, because S3/GCS are cost/performance optimized for around megabyte objects) is consistency: ensuring a reader sees a consistent set of blocks, i.e. all of the same database version.
Under concurrency, but also under a crash.
Similarly, the big issue with locks (exclusive or reader/writer) is not so much safety, but liveness.
It seems a bit silly to need to have (e.g.) rollback journals on top of object storage like S3.
Similarly, the problem with versioning is that for both S3/GCS you don't get to ask for the same version of different objects: there's no such thing.
So you'd need some kind of manifest, but then you don't need versioning, you can use object per version.
Under concurrency, but also under a crash.
Similarly, the big issue with locks (exclusive or reader/writer) is not so much safety, but liveness.
It seems a bit silly to need to have (e.g.) rollback journals on top of object storage like S3.
Similarly, the problem with versioning is that for both S3/GCS you don't get to ask for the same version of different objects: there's no such thing.
So you'd need some kind of manifest, but then you don't need versioning, you can use object per version.