Re:Endatabas: A SQLite-inspired, SQL document database with ...
by lichtenberger on Friday December 01, 2023 @06:03PM
(#38493791)
I'm working on something similar for the JVM, however with no document semantics, but on a much more fine granular level.
JSON is shredded during an initial import into a tree structure with fine granular nodes. Thus, an import can be done with very low memory consumption (permitted that auto-commit issues a sync to disk before RAM space is exceeded). Furthermore, it doesn't require a WAL for consistency. Instead the indexes are stored in a log-structure using a persistent tree (as in every commit creates a new tree root). A sliding snapshot algorithm makes sure, that only a fragment of a page has to be copied on a write.
As thus, it's also a perfect candidate for an event store, storing both the (lightweight) snapshots and tracking the changes optionally.
JSON is shredded during an initial import into a tree structure with fine granular nodes. Thus, an import can be done with very low memory consumption (permitted that auto-commit issues a sync to disk before RAM space is exceeded). Furthermore, it doesn't require a WAL for consistency. Instead the indexes are stored in a log-structure using a persistent tree (as in every commit creates a new tree root). A sliding snapshot algorithm makes sure, that only a fragment of a page has to be copied on a write.
As thus, it's also a perfect candidate for an event store, storing both the (lightweight) snapshots and tracking the changes optionally.
https://github.com/sirixdb/sirix
The architecture is described over here:
https://sirix.io/docs/concepts.html
Furthermore I'm working on a tutorial for a local client usage (work in progress):
https://sirix.io/docs/jsoniq-tutorial.html
Kind regards
Johannes